tech.safesquid.com

Posted: **Sun Nov 06, 2022 1:51 pm**

Client Scenario:
I want to have all private categories in a separate file with all the list of websites each category has.
Example: Private categories Company Blacklist, Company Whitelist, Admins etc.
Company Blacklist has URL’s: rediff.com, yahoo.com, Instagram.com facebook.com
Category file Company Blacklist should have the list of all the unique URL’s, all private category should be create in the same manner.
getcatlist.sh shell script is used to achieve the above-mentioned task.

getCatList.sh uses private_categories_db file to get lists of all private category and its corresponding URLs.
getCatList.sh can also use private_categories_db file as argument, if the private_categories_db file is not provide then it use the default file locate which is /tmp/safesquid/private_categories_db

How to get getCatList.sh shell script
Use below command to download and extract the shell script in /usr/local/src directory

Code: Select all

curl  https://tech.safesquid.com/forum/download/file.php?id=1161 -o /usr/local/src/getCatList.tar.gz ; tar -xzvf /usr/local/src/getCatList.tar.gz -C /usr/local/src/ ; chmod 755 /usr/local/src/getCatList.sh

How to Use getCatList.sh
Use below command to execute the shell script

Code: Select all

/usr/local/src/getCatList.sh

Output of getCatList.sh will be located at : /tmp/safesquid/safesquidPvtCategoryList/
Example:

Code: Select all

root@p-lab:~# ls -lrt /tmp/safesquid/safesquidPvtCategoryList/
total 571
-rw-rw-r-- 1 ssquid root     10 Nov  6 18:17 'Anonymous sites block'$'\n''Anonymous sites block'
-rw-rw-r-- 1 ssquid root     20 Nov  6 18:21 'MicrosoftteamApp'$'\n''MicrosoftteamApp'$'\n''MicrosoftteamApp'$'\n''MicrosoftteamApp'
-rw-rw-r-- 1 ssquid root     14 Nov  6 18:21 'phishing'$'\n''phishing'$'\n''phishing'
-rw-rw-r-- 1 ssquid root     20 Nov  6 18:21 'phishing'$'\n''phishing'
-rw-rw-r-- 1 ssquid root      8 Nov  6 18:21 'webproxy'$'\n''webproxy'
-rw-rw-r-- 1 ssquid root     12 Nov  6 18:21  Weapons
-rw-rw-r-- 1 ssquid root     57 Nov  6 18:21  Violence
-rw-rw-r-- 1 ssquid root     65 Nov  6 18:21  video
-rw-rw-r-- 1 ssquid root     21 Nov  6 18:21 'Unknown'$'\n''Unknown'
-rw-rw-r-- 1 ssquid root     27 Nov  6 18:21  UBS
-rw-rw-r-- 1 ssquid root     63 Nov  6 18:21  Translators
-rw-rw-r-- 1 ssquid root     24 Nov  6 18:21  test
-rw-rw-r-- 1 ssquid root     53 Nov  6 18:21  Sports
-rw-rw-r-- 1 ssquid root     42 Nov  6 18:21  socialnetworks
-rw-rw-r-- 1 ssquid root     31 Nov  6 18:21 'Restaurants &amp; Dining'
-rw-rw-r-- 1 ssquid root     25 Nov  6 18:21  Religion
-rw-rw-r-- 1 ssquid root     11 Nov  6 18:21  REDIFF
-rw-rw-r-- 1 ssquid root     27 Nov  6 18:21  rediff
-rw-rw-r-- 1 ssquid root     29 Nov  6 18:21  photosonline
-rw-rw-r-- 1 ssquid root     37 Nov  6 18:21 'Phishing &amp; Fraud'
-rw-rw-r-- 1 ssquid root     37 Nov  6 18:21  onlineshop
-rw-rw-r-- 1 ssquid root   1219 Nov  6 18:21  Malware
-rw-rw-r-- 1 ssquid root     30 Nov  6 18:21  filehosting
-rw-rw-r-- 1 ssquid root    301 Nov  6 18:21 'ALLOWED SITES BLOCKED FROM WRONG CATEGORY'
-rw-rw-r-- 1 ssquid root     67 Nov  6 18:21 'Bypass SSL Inspection'
-rw-rw-r-- 1 ssquid root    164 Nov  6 18:21  blogs
-rw-rw-r-- 1 ssquid root     94 Nov  6 18:21  Arts
-rw-rw-r-- 1 ssquid root   4092 Nov  6 18:21 'Anonymous sites block'
-rw-rw-r-- 1 ssquid root     33 Nov  6 18:21 'Alcohol &amp; Tobacco'
-rw-rw-r-- 1 ssquid root    121 Nov  6 18:21 'Advertisements &amp; Pop-Ups'
-rw-rw-r-- 1 ssquid root     12 Nov  6 18:21  ads

Example: Private category contents

Code: Select all

live.com
nexus.officeapps.live.com
officeapps.live.com
windows.com
Bypass\ SSL\ Inspection (END)

You can find getcatlist.sh shell script attached in this post.

getCatlist.tar.gz: (945 Bytes) Downloaded 91 times

tech.safesquid.com

Script: getCatList.sh

Script: getCatList.sh