As part of my study to prove that TCPtune works, I discovered these oddities in the Win98 TCP stack.
The first problem is really more of a bug than an oddity. It occurs when you have window scaling enabled and the window size above 65535 bytes. At this point win98 is forced to use window scaling. When this occurs windows 98 does a funny thing as it negotiates a connection. It doesn't send a window size in the initial SYN packet. The window size is sent as 0 and the wscale is set to the appropriate value. The reason it does this, we believe, is due to the possibility that the window scaling is rejected in which case the window size is invalid. Unfortunately windows forgets to send the new window size immediately after the connection is confirmed (this would result in a much smaller delay, approx. RTT). In some cases this is ok because the TCP stack has data to send and sends the window size with that data. However, in many situations (FTP's data connection for example) the remote side sends data first. This causes the connection to hang until the remote side sends a zero-window size verify packet (as described on page 42 of RFC 793). Basically this packet is one byte of data. The ack of such a byte of data includes the new window size. This is a method of flow control but it shouldn't be used to initiate a connection. In the case below this caused a one minute delay before data was sent. However, a two minute delay is recommended by the RFC. The final effect of this problem is that the connection will show poor performance.
16:50:02.608959 nai-a-ncrn.ftp-data > dyn40.caida.org.1026: S 2036105311:2036105311(0) win 16384 < mss 1460,nop,wscale 0,nop,nop,timestamp[|tcp] > (DF) [tos 0x8] (ttl 59, id 50587) 16:50:02.609283 dyn40.caida.org.1026 > nai-a-ncrn.ftp-data: S 131583:131583(0) ack 2036105312 win 0 < mss 1460,nop,wscale 2,nop,nop,timestamp[|tcp] > (DF) (ttl 128, id 4096) 16:50:02.666102 nai-a-ncrn.ftp > dyn40.caida.org.blackjack: . ack 41 win 17376 < nop,nop,timestamp 427151 1199 > (DF) [tos 0x10] (ttl 59, id 50588) 16:50:02.682285 nai-a-ncrn.ftp-data > dyn40.caida.org.1026: . ack 1 win 17376 < nop,nop,timestamp 427151 0 > (DF) [tos 0x8] (ttl 59, id 50589) 16:50:02.682489 nai-a-ncrn.ftp > dyn40.caida.org.blackjack: P 31:106(75) ack 41 win 17376 < nop,nop,timestamp 427151 1199 > (DF) [tos 0x10] (ttl 59, id 50590) 16:50:02.784783 dyn40.caida.org.blackjack > nai-a-ncrn.ftp: . ack 106 win 32707 < nop,nop,timestamp 1202 427151 > (DF) (ttl 128, id 4352) 16:51:02.414025 nai-a-ncrn.ftp-data > dyn40.caida.org.1026: . 1:2(1) ack 1 win 17376 < nop,nop,timestamp 427270 0 > (DF) [tos 0x8] (ttl 59, id 50820) 16:51:02.579099 dyn40.caida.org.1026 > nai-a-ncrn.ftp-data: . ack 2 win 32767 < nop,nop,timestamp 1800 427270 > (DF) (ttl 128, id 4608) 16:51:02.655687 nai-a-ncrn.ftp-data > dyn40.caida.org.1026: . 2:1450(1448) ack 1 win 17376 < nop,nop,timestamp 427271 1800 > (DF) [tos 0x8] (ttl 59, id 50821) |
The next oddity is in regards to the send window under win98. Under Unix the send window co-governs the amount of data that can be in flight without an acknowledgment. Well, under windows the send window is basically unused and the only thing that governs how much data is in flight is the current receive window size and the amount of data that is ready to send. I don't think I would call this a bug as it actually seems like a smart approach to me. However, it does mean that the sender window size used in v1.0 of TCPtune has no effect. It also means that throughput programs that base their measurements on varying sender window size have no effect when the data sender is on win98. However, I would like to stress that this is closer to what I would expect from a TCP stack. It only sends as much data as it has room to keep track of on the sender side. Then the performance is dictated by memory constraints instead of system configuration. Anyway, an example of is shown below. I ran HWB's throughput tests with a sender window size of 8kB. During this test, the TCP stack had more data in transit at one time than the sender window size would allow.
16:31:41.634908 nai-a-ncrn.56117 > dyn49.caida.org.blackjack: . ack 20273 win 32783 < nop,nop,timestamp 1571538 2251 > (DF) (ttl 59, id 46874) 16:31:41.636425 dyn49.caida.org.blackjack > nai-a-ncrn.56117: . 26065:27513(1448) ack 1 win 32768 < nop,nop,timestamp 2252 1571538 > (DF) (ttl 128, id 6912) 16:31:41.638010 dyn49.caida.org.blackjack > nai-a-ncrn.56117: . 27513:28961(1448) ack 1 win 32768 < nop,nop,timestamp 2252 1571538 > (DF) (ttl 128, id 7168) 16:31:41.639589 dyn49.caida.org.blackjack > nai-a-ncrn.56117: . 28961:30409(1448) ack 1 win 32768 < nop,nop,timestamp 2252 1571538 > (DF) (ttl 128, id 7424) 16:31:41.641181 dyn49.caida.org.blackjack > nai-a-ncrn.56117: . 30409:31857(1448) ack 1 win 32768 < nop,nop,timestamp 2252 1571538 > (DF) (ttl 128, id 7680) 16:31:41.642757 dyn49.caida.org.blackjack > nai-a-ncrn.56117: . 31857:33305(1448) ack 1 win 32768 < nop,nop,timestamp 2252 1571538 > (DF) (ttl 128, id 7936) 16:31:41.707323 nai-a-ncrn.56117 > dyn49.caida.org.blackjack: . ack 23169 win 32783 < nop,nop,timestamp 1571539 2252 > (DF) (ttl 59, id 46875) |
I got a lot of help in figuring this stuff out from Matt Mathis, Bill Fenner, and Jamshid Mahdavi. I also got a lot of information about 0 window sizes from Steven's books on networking as well as RFC 793. I submitted this to microsoft and it is supposedly going to be fixed in the release of win2000 and a near future patch of win98. However, there was no official comment.