Page 1 of 1

Dev Chat: 02/25/2014 Breaking the Bitstream

Posted: Wed Feb 26, 2014 7:07 am
by John Adams
These are excerpts from a very long chat between the devs about tearing down the bitstream - inclusing Xen's C# app and working with Scat on defining the 0x1 headers.

[quote]Session Start: Tue Feb 25 00:04:36 2014
[00:04] <@Xinux> well has some good info on movement but not what i'm looking for lol
[00:04] <@Xinux> http://udn.epicgames.com/Three/NetworkingOverview.html
[00:07] <Xen> OKay, I'm headed to bed. I'll see what I can dig up tomorrow
[00:07] <@Xinux> alright night.. hey think i can grab that program you used to spit out the data
[00:08] <Xen> Yeah, one sec, let me finish the menu real quick and I'll zip up the src

[00:09] <@Xinux> bah explain it LOL
[00:09] <@Xinux> The Unreal Wire Protocol defines the contents of packets as a stream of bits. The protocol deals with a variety of concepts, such as replicating variables and functions, creating actors, destroying actors, transmitting files, and exchanging packets reliably and unreliably (both types are used in various circumstances).
[00:09] <@Xinux> The protocol will undergo some major changes in the coming months to improve efficiency and reliability, so it would not be useful to document it here and now.
[00:10] <Xen> Yes, please document it!!

[00:11] <Xen> I'm putting the menu in for my sake. I kept having to comment/uncomment when I wanted to run something different.
[00:13] <Xen> where do you want me to dump it
[00:14] <@Xinux> if it's not big just upload it as a attachment on the forums in the spawn thread'
[00:19] <Xen> viewtopic.php?f=7&t=85&p=1214#p1214

Note: Xen explains how to use his program - see current detail in the above thread

[02:22] <@Xinux> ok so took my very last client spawn & movement packet
[02:22] <@Xinux> -- Client Spawn & Movement --
[02:22] <@Xinux> 2/25/2014 00:56:42
[02:22] <@Xinux> 192.168.1.101 -> 69.174.203.15
[02:22] <@Xinux> 0000: 00 09 01 72 01 00 16 00 00 00 EC 00 08 50 08 D2 ...r.........P..
[02:22] <@Xinux> 0010: 9A 19 72 42 FF 61 2B FA 47 A3 C6 B0 EF 0E 00 20 ..rB.a+.G......
[02:22] <@Xinux> cord 23311 8168 36166
[02:29] <@Xinux> Offset=78, dt=125, bVal=15, sVal=23311, iVal=1070684943
[02:29] <@Xinux> Offset=95, dt=86, bVal=232, sVal=8168, iVal=445456360
[02:29] <@Xinux> Offset=112, dt=71, bVal=70, sVal=36166, iVal=-547254970
[02:29] <@Xinux> .
[02:29] <@Xinux> fort a test i moved and did another packet
[02:30] <@Xinux> -- Client Spawn & Movement --
[02:30] <@Xinux> 2/25/2014 01:27:04
[02:30] <@Xinux> 192.168.1.101 -> 69.174.203.15
[02:30] <@Xinux> 0000: 00 09 00 39 01 00 16 00 00 00 1F 00 08 50 08 D2 ...9.........P..
[02:30] <@Xinux> 0010: 5C 8F FE 40 5F B1 29 0D C8 D0 C7 10 20 0D 00 20 \..@_.)..... ..
[02:30] <@Xinux> cord 19850 8244 36769
[02:30] <@Xinux> Offset=78, dt=125, bVal=138, sVal=19850, iVal=1080642954
[02:30] <@Xinux> Offset=95, dt=83, bVal=52, sVal=8244, iVal=524492852
[02:30] <@Xinux> Offset=112, dt=72, bVal=161, sVal=36769, iVal=1075941281

[06:15] <Xen> interesting that the dt is different though. I would expect those to be the same
[06:16] <Xen> although it looks like the first coord is dt 125, so maybe that is the dt for coords and the other 2 follow it.
[06:17] <Xen> And look at the difference between the offsets 95-78=17 and 112-95=17
[06:17] <Xen> You might be onto something there.
[06:39] <~john> morning. exciting stuff here Gonna start grabbing these IRC chats and posting them in the Dev Chats forum.
[06:44] <Xen> morning john
[06:49] <~john> excellent work on that app, btw. haven't downloaded it yet, but i see the output. That's cool.
[06:58] <Xen> It's very buggy right now because I'm doing no error checking =)
[07:00] <Xen> I'm fixing some of that now and will update my post when I've got a new version

[08:59] <Theeper> woohoo, time to crunch some packets
[09:09] <~john> morning Theeper
[09:09] <Theeper> good morning
[09:45] <Xen> got my program reading the first 100 bits of the bitstream and spitting out values for 1-bit, 3-bit, 10-bit and 14-bit
[09:45] <Xen> Hopefully that will let us try an figure out how the data from Amadeus' post is organized[/quote]

Re: Dev Chat: 02/25/2014 Breaking the Bitstream

Posted: Wed Feb 26, 2014 7:24 am
by John Adams
Where things got exciting

[quote][15:48] <Xen> hey scat
[15:51] <~john> he has been known to spontaneously fall asleep.
[15:51] <Xen> lol
[15:52] <Xen> Was wondering if he actually figured out the header info
[15:54] <~john> you mean the stuff before the 1's and 0's?
[15:54] <~john> i thought you had that already, too
[15:56] <Xen> I read it, but it doesn't match. Like in the Torlok packet, data length = 1856, but there are only ~1340 bits in the whole packet
[15:58] <Xen> and it shouldn't count the header bits, so it's even shorter
[15:58] <Xen> So like Amadeus feared, the order is probably different
[16:00] <~john> what does that mean exactly? they randomize the bits? lol
[16:02] <Xen> no, like in his post, first comes the packet id (14 bits), then 1 bit for ack/bunch, then 1 bit for create/destroy etc
[16:03] <Xen> But in Vanguard it could be packetid, ack/bunch, datalength
[16:03] <Xen> basically a different order, or even different size
[16:03] <~john> oh i see. but once we figure it out, it's the same for them all.
[16:03] <Xen> yeah should be
[16:08] <Xen> I figured out a few things like what indicates that a name data blob is coming, compared to a title data blob
[16:08] <Xen> I need to upload my latest code to share that with everyone else
[16:09] <Xen> got suffix and prefix figured out as well

[16:24] <@Xinux_Work> wonder how much they modified the unreal source tho
[16:24] <@Xinux_Work> "Out.GetNumBits()<=MAX_PACKET_HEADER_BITS",
[16:24] <@Xinux_Work> "C:\\Dev\\Vanguard\\Live\\Code\\External\\Unreal\\Engine\\UnConn.cpp",
[16:24] <~john> i thought we were pretty confident it would be quite modified? but the source would put us on the right track, only.

[16:49] <Xen> New code in the spawn packets thread
[16:49] <Xen> with instructions lol
[16:50] <~john> cool, thank you
[16:54] <~john> so you are taking the next value, and presenting possible dt's (byte, short, int, long) before moving to the next byte? so we can see patterns?
[16:55] <Xen> the dt I'm reading is always 7-bit
[16:55] <Xen> so for instance, offset 275, dt = 59
[16:55] <Xen> This means that starting at bit 275, I read 7 bits, whose value is 59
[16:56] <Xen> then after 7 bits, (783) I read in a bit
[16:56] <Xen> then I got back to 783 and read a byte
[16:56] <Xen> then back to 783 and read a short
[16:56] <Xen> then back to 783 and read an int
[16:56] <~john> gotcha. so why not 8 bits? a full byte? is that how bits work?
[16:57] <Xen> Well because it was exactly 7 bits between the end of the name and the start of the title for some npc's
[16:57] <Xen> And that patterns seemed to play out but it could be wrong
[16:58] <Xen> Xinux sent me a packet with Prefix, Name and Suffix and each section was 7 bits apart
[16:58] <Xen> remember, this is a bitstream which means you don't have to use full bytes
[16:59] <Xen> that's really the benefit of this, you can squeeze more in less
[16:59] <Xen> But this output you're seeing. That's what Xinux was referring to this morning when trying to find the NPC's location
[17:00] <~john> awesome.
[17:00] <Xen> I need to figure out how to spit out a float value in addition to the short, int, etc

[18:52] <@Xinux_Work> my one account is grandfathered in
[18:52] <@Xinux_Work> Membership level = 0, grandfathered = 1
[18:52] <~john> pretty sure my main acct is too. 3/12
[18:52] <@Xinux_Work> membership level 2 is gold member
[18:53] * john snickers like a 12 y/o

[20:27] <@Xen> evening gents
[20:32] <@Xen> So john, do you think I need to have my program read the entire log and spit out all that data?
[20:33] <~john> No, I don't think so. If we can suck one packet out, paste into a text file and read it, that should be good for now.
[20:34] <~john> what I may ask to do though is integrate (aka, steal) your bitstream explosion techniques when we build our packet parser. >then< we'd read the log, blow apart the packet, and stuff into the DB.
[20:34] <@theFoof> just my two cents but I think the best thing would be to figure out someway to convert the bitstream packets to hex, don't know if it's possible or not
[20:35] <~john> you mean during Collecting foof?
[20:35] <@theFoof> yeah
[20:35] <~john> Agreed. but if I can save the logs we already have, woot.
[20:35] <@Xen> Take any of my code you want john. The only thing is no one is checking that it's correct =)
[20:35] <@Scatman> good morning
[20:35] <@Xen> Foof, so basically anything in the bitstream that is 8 or less bits, read into a byte?
[20:36] <@Xen> anything between 9-16 bits, read into a short...and so on?
[20:36] <@Scatman> you can read them into anything bigger
[20:36] <@Xen> yeah
[20:36] <@theFoof> right - I haven't looked into the bitstream in detail (been to busy) but yeah just convert the actual data from the bits into their byte counterpart
[20:37] <@theFoof> would be ideal for analyzing I think, but again don't know if it's possible.
[/quote]

Re: Dev Chat: 02/25/2014 Breaking the Bitstream

Posted: Wed Feb 26, 2014 7:26 am
by John Adams
Here's where it got exciting :)

[quote]
[20:38] <@Xen> So scat, you are seeing the same thing I was on the bitstream header, where the data length is incorrect?
[20:39] <@Scatman> i'm reading your post now
[20:40] <@Xen> but in reference to your program/post
[20:41] <@Xen> you have data legnth =1856 for torlok
[20:41] <@Xen> but the packet is less than that big
[20:42] <@theFoof> doesn't that just mean the next packet for that channel contains the rest of that data?
[20:42] <@Xen> not sure, I thought it was meant to show how much data was in the bistream in that packet
[20:42] <@Scatman> it's a possibility
[20:44] <@theFoof> you're probably right, that wouldn't make sense I guess because the next header would have data length as well
[20:44] <@Xen> I don't think so though because of this code in the unreal source
[20:44] <@Xen> INT BunchDataBits = Reader.ReadInt( UNetConnection::MaxPacket*8 );
[20:45] <@Xen> that reads in the data length
[20:45] <@Xen> then a little further down it calls
[20:45] <@Xen> Bunch.SetData( Reader, BunchDataBits );
[20:45] <@Xen> which does:

Code: Select all

void FBitReader::SetData( FBitReader& Src, INT CountBits )
{
 	Num        = CountBits;
 	Pos        = 0;
 	ArIsError  = 0;
 	Buffer.Empty();
 	Buffer.Add( (CountBits+7)>>3 );
 	Src.SerializeBits( &Buffer(0), CountBits );
[20:45] <@Xen> that's all before then next packet is recieved I believe
[20:47] <@Xen> and look at this comment shortly after the SetData function is called:
[20:47] <@Xen> * // Bunch claims it's larger than the enclosing packet.
[20:50] <@Scatman> yeah the length of my torok data is 1295 it seems
[20:51] <@Xen> yup
[20:51] <@Xen> That's why I had my program output the first 100 bits with correspoding types of data so I could try to see a pattern where the datalength might be
[20:52] <@Xen> Only problem is that some of the header fields a variable. They may not always be there
[20:52] <@Xen> are variable*
[20:53] <@Xen> This is why I was hoping Amadeus could run through that function in the disassembled client. Might give hints on what's being read and in what order
[20:58] <@Scatman> yeah. the present/not present flags make this difficult
[21:02] <@Xen> I'm attempting to do it with smaller packets like the ones near when you first log on, with the challenge text
[21:02] <~john> Xen, did you see Amadeus' new post?
[21:02] <@Xen> 0000: 00 00 02 10 00 81 8C 98 21 A4 ..9...........!.
[21:02] <@Xen> 0010: 20 26 A6 22 A7 A3 22 10 AB 22 A9 9E 99 98 9A 18 &."..".."......
[21:02] <@Xen> 0020: 90 21 A4 20 26 A6 22 A7 A3 A2 9E 96 98 99 1B 1B .!. &.".........
[21:02] <@Xen> 0030: 18 1A 9C 1C 9A 19 90 29 AA 20 AA A9 1E 18 80 .......). .....
[21:02] <@Xen> Nope, I'll take a look
[21:03] <~john> you should put a request to him directly for whatever you need him to do. he's not around much, and just "skims" posts.
[21:03] <~john> maybe a PM.
[21:06] <@Xen> I'm responding to his post, but maybe he'll respond to a PM faster
[21:08] <@Xen> scat, I'm pretty much convinced that the first bits are the packetid, just like the source code does
[21:08] <@Scatman> yep
[21:08] <@Xen> 14-bits
[21:08] <@Scatman> INT PacketId = MakeRelative(Reader.ReadInt(MAX_PACKETID),InPacketId,MAX_PACKETID);
[21:08] <@Xen> But 12-bits works as well
[21:08] <@Xen> yep
[21:09] <@Scatman> even smaller may work, depending on what the actual number is
[21:09] <@Scatman> MAX_PACKETID is 16384
[21:09] <@Xen> yeah, so next should be ack/bunch
[21:09] <@Scatman> 1 << 14 = 16384
[21:09] <@Scatman> so 14 bits
[21:09] <@Xen> yep
[21:11] <@Scatman> // Disassemble and dispatch all bunches in the packet.
[21:11] <@Scatman> that comment is interesting
[21:12] <@Xen> ahh, so maybe more than one bunch per packet
[21:12] <@Scatman> yeah
[21:12] <@Scatman> SOE will combine as many packets as they can within their max packet size
[21:12] <@Scatman> just like tcp does
[21:14] <@Xen> so, try putting this one in your program. It's a small packet
[21:14] <@Xen> 0000: 01 00 12 00 00 00 7C 43 94 70 06 C7 26 34 25 B1 ......|C.p..&4%.
[21:14] <@Xen> 0010: E3 80 00 55 47 0A B4 C0 ...UG...
[21:14] <@Xen> I get packetid = 892
[21:15] <@Xen> isAck = 1
[21:15] <@Xen> source code then says if isAck, read another 14-bit integer
[21:15] <@Xen> I'm getting 8488 for that
[21:18] <@Scatman> it could be correct
[21:18] <@Scatman> it's just a sequential number
[21:18] <@Xen> but there's more data
[21:18] <@Scatman> unless it's reallllyy early in the session?
[21:19] <@Xen> I pulled it from this Flarney Jones packet on the svn. Looks like about 1/2 through.
[21:21] <@Xen> so the 8488 is the ack packet id. I'm assuming that is the server telling the client it got packet 8488
[21:21] <@Scatman> yep
[21:21] <@Xen> does it make sense that the server is sending packet 893 to acknowledge packet 8488?
[21:22] <@Scatman> is the 8488 from client to server or vice versa?
[21:22] <@Xen> that whole packet is server-->client
[21:23] <@Scatman> then since it's an ack from the server, the client should've sent a packet with Packet ID 8488 before that ack
[21:23] <@Xen> yeah, let me go up a few lines to see the last 01 packet the client sent

[21:27] <@Scatman> i wonder what the size of UNetConnection::MaxPacket is
[21:27] <@Scatman> the value, rather
[21:27] <@Xen> Yes, I couldn't figure that out
[21:27] <@Xen> It's initialized to 0 in the constructor
[21:27] <@Scatman> it's negotiated in an early packet somewhere
[21:27] <@Scatman> just like in VG
[21:28] <@Xen> I grepped it but couldn't find any code where it was being set
[21:28] <@Xen> oh I see, it's based on packet data
[21:29] <@Scatman> yep, unfortunately
[21:29] <@Scatman> for us :P
[21:29] <@Xen> I just used 12-bit because that's what was in Amadeus initial post
[21:30] <@Scatman> hmmmm
[21:30] <@Scatman> what is your condition for getting the channel type?
[21:30] <@Scatman> the if statement
[21:30] <@Scatman> in your C# prog
[21:30] <@Scatman> because i think amadeus has it wrong
[21:31] <@Xen> I didn't use his post for the logic. I used the source code
[21:31] <@Scatman> ok
[21:31] <@Xen>

Code: Select all

                 if (uBunch.bReliable != 0 || uBunch.bOpen != 0)
                     uBunch.chType = reader.readInt(CHTYPE_MAX);
                 else
                     uBunch.chType = CHTYPE_None;
[21:31] <@Scatman> i used his logic. i changed it to the source's logic and now i'm getting the channel type, which is changing my data size
[21:31] <@Xen> really?
[21:32] <@Scatman> i'm getting a data length of 232 now
[21:32] <@Xen> hmm
[21:32] <@Scatman> that first batch of 1's and 0's is 236 bits
[21:32] <@Xen> we talking about torlok?
[21:32] <@Scatman> yeah
[21:33] <@Xen> so first 14-bit = 236?
[21:33] <@Scatman> so if we break it up 232 bits later, let's see if we get another header
[21:33] <@Xen> packetid =236?
[21:34] <@Scatman> no sorry. i'm just saying there are 236 bits between the data length and the name length
[21:34] <@Scatman> oh it's actually 241.. hang on. let me make sure i'm not miscalculating :P
[21:35] <@Xen> lol
[21:35] <@Xen> that seems a bit tight though. data len = 232, then name len comes right at 241
[21:36] <@Xen> This is why I need someone checking my work =)
[21:36] <@Scatman> oh so it's 233 bits...
[21:37] <@Scatman> so that'd only leave 1 bit between the first "packet" and the name length
[21:37] <@Scatman> which wouldn't make sense?
[21:38] <@Xen> that's what I think, unless data len is an offset to where the data starts?
[21:38] <@Xen> But that doesn't make much sense to me
[21:41] <@Xen> what are you getting for the channel type in that packet?
[21:42] <@Scatman> 0
[21:42] <@Scatman> same for you?
[21:43] <@Xen> same, which is supposedly an invalid channel
[21:44] <@Xen> check this out though....change channel index to a 12 bit integer and then channel type becomes 2, which is actor channel
[21:44] <@Xen> then data len = 1024
[21:46] <@Scatman> hmmm
[21:47] <@Xen> I'm checking rahlus packet with same logic to see what comes out there
[21:47] <@Scatman> kk
[21:48] <@Xen> same thing, ch type = 2, data len = 1024
[21:48] <@Xen> different channel index as we would expect

[21:50] <@Xen> going to check another npc packet
[21:50] <@Scatman> channel index to a 12 byte integer?
[21:51] <@Xen> 12-bit
[21:52] <@Xen> pass in 4095 to the readInt function
[21:53] <@Scatman> weird.. i'm getting..
[21:54] <@Scatman> Channel number = 309
[21:54] <@Scatman> Sequence number = 2
[21:54] <@Scatman> Channel type = 0
[21:54] <@Scatman> Data length: 2106
[21:54] <@Scatman> integer = bit_reader_int(&reader, 4095);
[21:54] <@Scatman> 153 printf("Channel number = %lu\n", integer);
[21:54] <@Xen> hmm
[21:54] <@Xen> I'm manually running through my file
[21:55] <@Xen> what are you calling to get the channel type
[21:56] <@Scatman> integer = bit_reader_int(&reader, 8);
[21:56] <@Scatman> 163 printf("Channel type = %lu\n", integer);
[21:57] <@Xen> so you are reading 4 bits right?
[21:57] <@Xen> This was throwing me off earlier
[21:57] <@Scatman> ah fuck no i'm reading 8
[21:57] <@Scatman> but that's an enum
[21:58] <@Scatman> but no, that shouldn't matter
[21:58] <@Xen> well 8 should be 4 bits right?
[21:58] <@Xen> 1000
[21:58] <@Xen> in binary
[21:58] <@Scatman> yes
[21:58] <@Scatman> 8 is my max value
[21:58] <@Xen> can you see what the 4 bits are, at that offset right before you ready in chan type?
[22:00] <@Scatman> hang tight
[22:00] <@Scatman> i never did that MakeRelative crap
[22:00] <@Xen> that shouldn't matter right?
[22:01] <@Xen> I'm not doing that in my code :P
[22:02] <@Scatman> yep, i just didn't know what it did
[22:03] <@Xen> so manually looked at a 3rd npc packet and it's coming out with the same thing ch index = 45, ch type = 2, data len - 1024
[22:03] <@Xen> maybe it's that 4095 call. Try 4096
[22:06] <@Xen> that max value thing is confusing me. I think max value is supposed to be 1 more than what that many bits can hold. So 12-bit = 111111111111 = 4095
[22:07] <@Xen> Scat fell asleep :(
[22:07] <@Scatman> yeah that's what it seems like
[22:07] <@Scatman> lol nope i'm here
[22:07] <@Xen> lol
[22:07] <@Xen> did you try 4096?
[22:08] <@Scatman> data len 2106 when 4095
[22:08] <@Scatman> anddd
[22:08] <@Scatman> same with 4096
[22:09] <@Xen> try channel type with 16 instead of 8
[22:09] <@Xen> 8 would be 3-bit, 111 = 7 + 1 = 8
[22:09] <@Xen> 4-bit = 1111 = 15 + 1 = 16
[22:10] <@Scatman> 4096 in channel index and 16 in channel type gives me
[22:10] <@Xen> damn lol
[22:11] <@Scatman> Channel number = 309
[22:11] <@Scatman> Sequence number = 2
[22:11] <@Scatman> Channel type = 0
[22:11] <@Scatman> Data length: 1053
[22:11] <@Scatman> wtf?
[22:11] <@Xen> oh shit, I missed sequence number :/
[22:11] <@Xen> But that could still make sense
[22:11] <@Xen> 1053
[22:11] <@Scatman> haha thank god i thought it was losing my mind
[22:12] <@Xen> try running Rahlsu
[22:13] <@Scatman> Channel number = 157
[22:13] <@Scatman> Sequence number = 2
[22:13] <@Scatman> Channel type = 0
[22:13] <@Scatman> Data length: 1441
[22:13] <@Xen> hmm, that's more than his packet len right
[22:14] <@Scatman> we need to figure out what these values are in vg client: MAX_CHANNELS, MAX_CHSEQUENCE, CHTYPE_MAX
[22:14] <@Scatman> or guess them right :P
[22:14] <@Xen> yep
[22:14] <@Xen> I would expect channel type to be something other than 0, wouldn't you?
[22:14] <@Scatman> yes
[22:15] <@Scatman> because it's being read in
[22:15] <@Xen> maybe they don't use sequence =)
[22:15] <@Scatman> if it wasn't being read in, 0 would be ok
[22:15] <@Scatman> HMMMMMM
[22:15] <@Scatman> there is a sequence on the VG packet itself...
[22:15] <@Scatman> why would they need 2?
[22:16] <@Xen> true, but it's still odd that the 3 npc's I tried all had 1024 for data len even when their names and titles were different lengths
[22:16] <@Scatman> get out UnConn.cpp::void UNetConnection::SendAck( INT AckPacketId, UBOOL FirstTime )
[22:17] <@Scatman> line 606
[22:17] <@Scatman> if (!InternalAck)
[22:17] <@Scatman> so it seems like they have a way of disabling acks
[22:17] <@Scatman> yeah that is kinda odd
[22:17] <@Xen> hmm
[22:18] <@Scatman> you saw an ack in the log though, right?
[22:18] <@Xen> I think we are getting close though. I'm pretty confident that we've got the first couple pieces down (packet id, isAck, create/destroy, and reliable)
[22:18] <@Xen> yeah I did
[22:18] <@Scatman> aye
[22:18] <@Scatman> it's these #defines that COULD be throwing it off
[22:19] <@Scatman> because VG could configure those differently
[22:19] <@Xen> yep
[22:20] <@Xen> I need to change my code to output the correct bit values :/
[22:20] <@Scatman> yeah thatd be a good idea for me too
[22:20] <@Scatman> also want to make it loop
[22:21] <@Xen> I think you got the max value right. You just do 1 << bit-num
[22:21] <@Xen> so 1 << 3 = 8 for 3-bit and so on
[22:22] <@Scatman> yah
[22:25] <@Xen> hmm, 32-bit wouldn
[22:25] <@Xen> wouldn't work for the ReadInt function as-is
[22:26] <~john> <@Xen> true, but it's still odd that the 3 npc's I tried all had 1024 for data len even when their names and titleswere different lengths
[22:26] <~john> could they be padding the stream withbogus data?
[22:26] <~john> just to keep a static length?
[22:27] <@Scatman> could be.
[22:28] <@Scatman> but i feel if they have on/off flags for 1 single bit, they wouldnt waste space with padding fixed length strings
[22:28] <@Scatman> but really, who knows
[22:28] <@Xen> yeah
[22:28] <@Scatman> i think part of our problem is, is that we are not splitting up the packet into multiple possible bunches
[22:29] <@Scatman> they are definitely doing that in their code
[22:30] <@Xen> yeah that was a good pick up
[22:31] <@Xen> But we should be able to verify that we've got the right data length if it starts a new header
[22:31] <@Scatman> yah
[22:31] <@Scatman> that is true
[22:31] <@Xen> scat, what do you think void UNetConnection::ReceivedRawPacket( void* InData, INT Count ) is used for in UnConn.cpp?
[22:32] <@Xen> I noticed it lowers the bitsize before it passed it to ReceivedPacket function
[22:34] <@Xen> Or does that function not even come into play in our case
[22:35] <@Scatman> oh wow
[22:35] <@Scatman> that is always called BEFORE ReceivedPacket
[22:35] <@Scatman> RecievedPacket is only called inside of ReceivedRawPacket
[22:36] <@Scatman> this is really interesting
[22:37] <@Xen> that last byte business isn't crc magic right?
[22:38] <@Scatman> not exactly sure what it's for yet

[23:10] <@Xen> I may have found the data len scat
[23:11] <@Scatman> yeah xen?
[23:12] <@Xen> offset 45 seems to be promising for the 3 npc's I'm looking at
[23:12] <@Scatman> ok
[23:12] <@Scatman> what sizes did u decrease?
[23:13] <@Scatman> or are you just running thru the packet header code?
[23:13] <@Xen> Ralhsu = 1384
[23:13] <@Xen> + 45 (offset) + 12 (12-bit data len) = 1441
[23:14] <@Xen> his bitstream len = 1447
[23:14] <@Xen> total bitstream len
[23:15] <@Xen> Torlok = 1287 + 45 (offset) + 12 = 1344
[23:15] <@Xen> his total bitstream len = 1351
[23:16] <@Xen> Talayah Khadem = 1038 + 45 + 12 = 1095
[23:16] <@Xen> her total bistream len = 1095
[23:17] <@Xen> I'm just manually looking through my output
[23:18] <@Scatman> hmmm ok
[23:18] <@Xen> Going to work backwards from offset 40
[23:18] <@Xen> err 45
[23:19] <@Scatman> send 'em again?
[23:19] <@Xen> In those files look at the bottom section of offsets where it outputs things in 1-bit, 3-bit, 4-bit....
[23:19] <@Xen> did you get both of them
[23:20] <@Scatman> just torlok
[23:21] <@Scatman> k thx
[23:22] <@Xen> working backwards, I got ch type = 2 (3 bits back to offset 42)
[23:25] <@Xen> there's about 5 bits that are floating when I work backwards
[23:26] <@Scatman> you mean like at the end?
[23:26] <@Xen> no, if I start at 45 meaning the datalen, I go back 3 bits to read the channel
[23:27] <@Xen> then go back 10 bits to read the ch seq
[23:27] <@Xen> then 10 bits back to get the ch index
[23:27] <@Xen> and so on
[23:28] <@Scatman> ahh i see
[23:28] <@Xen> but I've got 5 bits between isAck and isReliable that I don't know where they belong
[23:30] <@Scatman> maybe a bigger packet id?
[23:30] <@Xen> I think it's close though. I got Rahlsu chan index = 260 and Torlok = 265. Torlok has higher packet id than Rahlsu also
[23:31] <@Xen> Could be. For some reason my 16-bit int for offset = 0 which doesn't make sense
[23:31] <@Xen> actualy it does make sense. I forgot to set it in code :(
[23:31] <@Scatman> heh :P
[23:32] <@Xen> It could be that open/close flags are always sent
[23:33] <@Xen> k 16-bit value on offset 1 is same as 14-bit
[23:34] <@Xen> notice how offset 7-15 = 0
[23:34] <@Xen> I bet that is the packet id
[23:34] <@Xen> from 0-15
[23:35] <@Xen> ugh, that makes bit 16 = 1 for isAck
[23:37] <@Scatman> does it?
[23:37] <@Scatman> 1 << 15
[23:37] <@Scatman> oh
[23:37] <@Scatman> damn
[23:37] <@Xen> ..whatcha got
[23:38] <@Scatman> nothin. accidentally did 15 isntead of 16 heh
[23:38] <@Scatman> i get an ack too
[23:38] <@Xen> lol
[23:41] <@Scatman> well it's almost 2am here, gotta get some sleep and think about it
[23:41] <@Scatman> we're getting closer thoug
[23:42] <@Xen> yeah I gotta sleep to
[23:42] <@Xen> later
[23:42] <@Scatman> night[/quote]