When using websites without sub-domains there are no categorization issue, everything works accordingly.
Example:
https://youtube.com belongs to a custom category YouTube.
- image.png (178.57 KiB) Viewed 1344 times
Add youtube.com to custom categories GOOGLE SERVICES, test_category1 and test_category2
- image.png (185.12 KiB) Viewed 1344 times
when searched for youtube.com from interface after updating, all custom category is visible.
- image.png (182.65 KiB) Viewed 1344 times
Validating from developer tools
- image.png (614.79 KiB) Viewed 1338 times
Validate using /tmp/safesquid/private_category_db from SafeSquid CLI
- image.png (67.58 KiB) Viewed 1344 times
Validate from
https://key.safesquid.com
- image.png (116.11 KiB) Viewed 1344 times
Comparing the logs of mail.google.com_for_test_category2.txt and youtube.com_for_test_category2.txt
below are the log lines which are not present in youtube.com_for_test_category2.txt log file
Code: Select all
Origin: https://safesquid.cfg
Connection: keep-alive
Referer: https://safesquid.cfg/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
2022 07 27 21:13:39.154 [656] header: debug: url_command_extract:168 connection: HTTP_REQUEST
2022 07 27 21:13:39.154 [656] header: debug: url_command_extract:168 connection: HTTP_PROXY
2022 07 27 21:13:39.154 [656] request: debug: url_command_parse:16 invalid url
2022 07 27 21:13:39.154 [656] network: debug: net_filebuf_read:2057 [timeout:6] start [bytes:111] client 192.168.2.155:8080 192.168.2.105:50682
2022 07 27 21:13:39.154 [656] network: debug: net_filebuf_read: client 192.168.2.155:8080 192.168.2.105:50682 [111.000B in 0.0090ms] [speed: 12.3333MBps] loops:1 BPR:111
2022 07 27 21:13:39.154 [656] request: debug: buffered: POSTDATA:111 bytes
2022 07 27 21:13:39.154 [656] interface: debug: handle_request:2298 [IP:192.168.2.105] []
2022 07 27 21:13:39.155 [656] interface: debug: interface_handle_request:908 connection->header->file:(/) connection->header->host_header(safesquid.cfg)
2022 07 27 21:13:39.155 [656] interface: debug: invoke_handler:89 [category_manager]
2022 07 27 21:13:39.155 [656] network: debug: DNS[IPv4]: cache[hit] category.safesquid.net -> 157.245.108.161
2022 07 27 21:13:39.155 [656] network: debug: connecting: category.safesquid.net 192.168.2.155:27311 ENOTCONN:0 [fd:49]
2022 07 27 21:13:39.155 [656] network: debug: protocol_start:238 net_connect(127.0.0.1 -> category.safesquid.net:443)
2022 07 27 21:13:39.180 [656] ssl: debug: LoadClientCertificates:929 /usr/local/safesquid/security/activation_key: reading certificate success: reading key: success
2022 07 27 21:13:39.180 [656] ssl: debug: Socket::EncryptS:1301 ENOTCONN:0 SSL_set_tlsext_host_name(category.safesquid.net)
2022 07 27 21:13:39.202 [656] network: connected: category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 [fd:49]
2022 07 27 21:13:39.224 [656] ssl: debug: Socket::EncryptS:1361 category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 (TLS_AES_256_GCM_SHA384)
2022 07 27 21:13:39.225 [656] ssl: debug: check_cert_dns:3873 certificate common name category.safesquid.net matches category.safesquid.net
2022 07 27 21:13:39.225 [656] header: debug: header_send: category.safesquid.net 192.168.2.155:27311 157.245.108.161:443
POST /categorydb/privatedb2.php HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows 98; Win 9x 4.90)
Content-Type: application/x-www-form-urlencoded
Content-Length: 124
Host: category.safesquid.net
Connection: keep-alive
Keep-Alive: timeout=6
2022 07 27 21:13:39.225 [656] header: debug: header_get_reconnect:1055 timeout(75) category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 category.safesquid.net
2022 07 27 21:13:39.892 [656] header: debug: response headers from category.safesquid.net 192.168.2.155:27311 157.245.108.161:443:
HTTP/1.1 200 OK
Date: Wed, 27 Jul 2022 15:43:39 GMT
Server: Apache/2.4.41 (Ubuntu)
Content-Length: 7
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: text/html; charset=UTF-8
2022 07 27 21:13:39.893 [656] request: debug: http_fetch_p:492 standard content body expected: https://category.safesquid.net:443/categorydb/privatedb2.php
2022 07 27 21:13:39.893 [656] network: debug: net_filebuf_read:2057 [timeout:75] start [bytes:7] category.safesquid.net 192.168.2.155:27311 157.245.108.161:443
2022 07 27 21:13:39.893 [656] network: debug: net_filebuf_read: category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 [7.000B in 0.0390ms] [speed: 179.4872KBps] loops:1 BPR:7
2022 07 27 21:13:39.893 [656] network: Socket::server disconnecting: category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 [xacts:1] [read:210.000B] [write:388.000B]
2022 07 27 21:13:39.893 [656] network: debug: ~Socket:617 s_shut category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 [SERVER|SSL|S_SSL_SHUT|S_NO_LINGER]
2022 07 27 21:13:39.893 [656] network: debug: ~Socket:656 s_shut category.safesquid.net 192.168.2.155:27311 157.245.108.161:443 [SERVER|SSL|READ_CLOSED|WRITE_CLOSED|S_SSL_SHUT|S_NO_LINGER]
2022 07 27 21:13:39.893 [656] header: debug: header_send: client 192.168.2.155:8080 192.168.2.105:50682
HTTP/1.1 200 OK
Date: Wed, 27 Jul 2022 15:43:39 GMT
Content-Type: text/xml
Content-Length: 101
Cache-Control: no-cache
Connection: keep-alive
Keep-Alive: timeout=6
X-Powered-By: safesquid-2022.0718.1917.3-swg-standard
X-SafeSquid-Client-ID: 656.2
X-SafeSquid-User-Groups: LocalProxy2
X-SafeSquid-Instance: IqbN6tiwnAxHzsYK
X-Registered-Domain: safesquid.cfg
2022 07 27 21:13:39.893 [656] request: debug: http_send_body:1052 101 bytes -> 192.168.2.105
2022 07 27 21:13:39.893 [656] network: debug: process_entry:1537 656.2 transaction speed: 739.2940ms
2022 07 27 21:13:41.923 [656] network: debug: process_entry:1681 Pipelined Request: 656.2
2022 07 27 21:13:41.924 [656] header: debug: header_get(client) 656.3: client 192.168.2.155:8080 192.168.2.105:50682:
POST / HTTP/1.1
Host: safesquid.cfg
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0
Accept: application/xml, text/xml, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 45
Origin: https://safesquid.cfg
Connection: keep-alive
Referer: https://safesquid.cfg/
Sec-Fetch-Dest: empty
Sec-Fetch-Mode: cors
Sec-Fetch-Site: same-origin
Pragma: no-cache
Cache-Control: no-cache
2022 07 27 21:13:41.924 [656] header: debug: url_command_extract:168 connection: HTTP_REQUEST
2022 07 27 21:13:41.924 [656] header: debug: url_command_extract:168 connection: HTTP_PROXY
2022 07 27 21:13:41.924 [656] request: debug: url_command_parse:16 invalid url
2022 07 27 21:13:41.924 [656] network: debug: net_filebuf_read:2057 [timeout:6] start [bytes:45] client 192.168.2.155:8080 192.168.2.105:50682
2022 07 27 21:13:41.924 [656] network: debug: net_filebuf_read: client 192.168.2.155:8080 192.168.2.105:50682 [45.000B in 0.0510ms] [speed: 882.3530KBps] loops:1 BPR:45
2022 07 27 21:13:41.924 [656] request: debug: buffered: POSTDATA:45 bytes
2022 07 27 21:13:41.924 [656] interface: debug: handle_request:2298 [IP:192.168.2.105] []
2022 07 27 21:13:41.924 [656] interface: debug: interface_handle_request:908 connection->header->file:(/) connection->header->host_header(safesquid.cfg)
2022 07 27 21:13:41.924 [656] interface: debug: invoke_handler:89 [category_manager]
2022 07 27 21:13:41.925 [656] header: debug: header_send: client 192.168.2.155:8080 192.168.2.105:50682
HTTP/1.1 200 OK
Date: Wed, 27 Jul 2022 15:43:41 GMT
Content-Type: text/xml
Content-Length: 480
Cache-Control: no-cache
Connection: keep-alive
Keep-Alive: timeout=6
X-Powered-By: safesquid-2022.0718.1917.3-swg-standard
X-SafeSquid-Client-ID: 656.3
X-SafeSquid-User-Groups: LocalProxy2
X-SafeSquid-Instance: IqbN6tiwnAxHzsYK
X-Registered-Domain: safesquid.cfg
2022 07 27 21:13:41.925 [656] request: debug: http_send_body:1052 480 bytes -> 192.168.2.105
2022 07 27 21:13:41.925 [656] network: debug: process_entry:1537 656.3 transaction speed: 1.1390ms
2022 07 27 21:13:45.993 [656] network: debug: process_entry:1681 Pipelined Request: 656.3
2022 07 27 21:13:45.993 [656] header: debug: header_get(client) 656.4: client 192.168.2.155:8080 192.168.2.105:50682:
POST / HTTP/1.1
Host: safesquid.cfg
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Firefox/102.0
Accept: application/xml, text/xml, */*; q=0.01
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate, br
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
X-Requested-With: XMLHttpRequest
Content-Length: 111
However, their are no differences in mail.google.com_for_test_category1 and youtube.com_for_test_category2 log files
I have attached logs of find_client_id.sh when adding custom category to mail.google.com and youtube.com
