My short answer?
Here is a long explanation with my data and model.
Since the end of January 2020, I have changed my V2Ray server settings from vmess over TCP to vmess over TLS + WebSocket.
I deployed the V2Ray server to two different cloud providers with different UUID, different IP, different ports, different domain names, different OS flavors, and different TLS certificates. The traffic data form Google Compute Engine (GCE) is for training and validation, while the one from Amazon Light Sail (LS) is for testing. As you can see from the traffic file below, there is a time gap in GCE data because there was a power outage in my neighborhood and my router didn’t hook up to a UPS. I didn’t realize it until a week later.
drwxr-xr-x 2 Ricky Ricky 16M Mar 1 16:31 tcpsorter.20200129.20200206.GCE.TLS.WS -rw-r--r-- 1 Ricky Ricky 1.4G Feb 29 23:35 tcpsorter.20200129.20200206.GCE.TLS.WS.tar.gz drwxr-xr-x 2 Ricky Ricky 34M Mar 1 16:23 tcpsorter.20200213.20200229.GCE.TLS.WS -rw-r--r-- 1 Ricky Ricky 3.3G Feb 29 23:38 tcpsorter.20200213.20200229.GCE.TLS.WS.tar.gz drwxr-xr-x 2 Ricky Ricky 14M Mar 11 17:58 tcpsorter.20200304.20200311.LS.TLS.WS -rw-r--r-- 1 Ricky Ricky 1.3G Mar 11 17:49 tcpsorter.20200304.20200311.LS.TLS.WS.tar.gz
Here is the summary of the data from GCE. I split 80% of the data for training and 20% of the data for validation:
Statistics: Total V2ray traffic 32005, Total non-V2ray traffic 346431 Output train traffic 51208, Total validation traffic 12802
Here is the summary of the data from LS. I used all of them to test the model trained by the GCE data. The previously trained model knows nothing about the data from LS. There is no data leak.
Statistics: Total V2ray traffic 10872, Total non-V2ray traffic 76316 Output train traffic 87188, Total validation traffic 0
When I trained the model with GCE data, I used the early stopping technique which minimizes the validation loss. Within the 1st epoch, it stops with 0.9999 accuracy.
Epoch 1/1 1600/1600 [==============================] - 603s 377ms/step - loss: 0.0068 - accuracy: 0.9981 - val_loss: 7.2060e-07 - val_accuracy: 0.9999
The ROC curve of validation data looks perfect.
Then, I collected data from AWS Light Sail for a week. I loaded the model to do inference only with the LS data. See the result below:
# Create evaluation generator eval_generator = PacketDataGenerator(eval_file_list, shuffle=False) eval_result = model.evaluate_generator(eval_generator,workers=3, use_multiprocessing=True, verbose=1) 2724/2724 [==============================] - 18s 6ms/step print(eval_result) [7.367991372575489e-08, 0.9997590780258179]
The ROC curve of test data looks perfect. The accuracy is 0.999759.
I didn’t release the Python notebook this time. Because there are nothing changes compared to the previous vmess over TCP notebook except the data are different. But I did upload the trained model to my Github repo.
Note that my non-V2Ray traffic contains a variety of traffic types that pass-through my home router. For example, the V2Ray server uses the port other than 443. So I can count the number of possible HTTPS traffic below:
[Ricky@gtx tcpsorter.20200304.20200311.LS.TLS.WS]$ ls -alh | grep "\.443\.bin$" | wc -l 70616
No, V2Ray with TLS can NOT blend in other HTTPS traffic. You are still exposed like using vmess over TCP.
I’m in the procrastination loop lately due to the pollen season. Even after taking non-drowsy anti-allergy medicine, I’m still sleepy. But I will keep up to look for a solution.