Final Item Parsing

[John Adams]

(hopefully)

First, let me tell you what I'm looking at. I recently re-parsed all our logs to get a unique count of every item_id belonging to a different "category" across different collected logs. Using these 3 values as my "unique" allows me to see, for example, item_id 1 in log_id 1 and how many different opcodes (categories) the item was seen in. For example:

item_id	name			category	log_id
19413	Tiny Snake Fang		Acquired	10
19413	Tiny Snake Fang		Bank		74
19413	Tiny Snake Fang		Buyback		13
19413	Tiny Snake Fang		Link		1039
19413	Tiny Snake Fang		Loot		10
19413	Tiny Snake Fang		Server		65

This 1 item was Acquired, sent to a Bank slot, sold to a Merchant (buyback), it was Linked in chat, Looted off a corpse, and was in player inventory when they logged in/chunked. There were 103 "unique" instances of this item_id across many logs, but all I cared about was that this Tiny Snake Fang did not have any additional properties on it that we cared about (of course trash loot won't, but this was a simple example).


Previous parsed items used only a unique of the item_id, so if the first Tiny Snake Fang had a player augment on it of +883 strength, that was the only version of that item we pushed to our database. What this new parsing allows me to do is evaluate all versions of this item and determine which one is best for our Items Database. So, that brings me to this post.


From the raw data I see now (in excess of 2,200,000 duplicated items) I need to know what order to parse items in that will give me the "purest" version of the item for our database. To explain how that works, each item Category is basically the opcode that was triggered to send the item packet. Below is a list of those opcodes and my best guess at what types of items the opcode handles.

Code: Select all

    /*** Pure items found in the wild ***/    /* Appears to be any reward item (Adventure/Craft/Harvest - count: 11,104 */    if (!ProcessItemPacketsByOpcode(OP_ServerLootList, "Loot"))        return false;    /* Also appears to be Loot - count: 13,497 */    if (!ProcessItemPacketsByOpcode(OP_ServerAcquireItem, "Acquired"))        return false;    /* Possibly Quest/Crafting results - count: 6,631 */    if (!ProcessItemPacketsByOpcode(OP_ServerQuestRewardItem, "Reward"))        return false;    /* Things acquired during game play - count: 2,651 */    if (!ProcessItemPacketsByOpcode(OP_ServerBoughtTradeItem, "Bought"))        return false;    /* Havesting resources - count: 101 */    if (!ProcessItemPacketsByOpcode(OP_ServerHarvestingPotentialYield, "Harvest"))        return false;    /* Loot from Chests? count: 1,846 */    if (!ProcessItemPacketsByOpcode(OP_ServerChestItems, "Chest"))        return false;    /* Merchant Inventory - count: 2,957 */    if (!ProcessItemPacketsByOpcode(OP_ServerMerchantItemMouseover, "Merchant"))        return false;    /* Items for sale on Telon Exchange - count: 1,935 */    if (!ProcessItemPacketsByOpcode(OP_ServerExchangeMouseoverItem, "Exchange"))        return false;    /* Items from the SOE Marketplace/Claim? count: 514 */    if (!ProcessItemPacketsByOpcode(OP_ServerMarketItem, "Market"))        return false;    /* Items linked in chat? count: 1,557 */    if (!ProcessItemPacketsByOpcode(OP_ItemLink, "Link"))        return false;    /* Necro Pet Toys - count: 53 */    if (!ProcessItemPacketsByOpcode(OP_ServerGrafts, "Graft")) /* Pet Augments? */        return false;    /*** Impure, player-altered items ***/    /* Crafted Items for Assembly - count: 509 */    if (!ProcessItemPacketsByOpcode(OP_ServerAssemblyItems, "Assembly"))        return false;    /* Assuming this is player inventory lists? count: 13,174 */    if (!ProcessItemPacketsByOpcode(OP_ServerItems, "Server")) /* Player inventory */        return false;    /* Items from Inspecting players - count: 2,979 */    if (!ProcessItemPacketsByOpcode(OP_ServerInspectItems, "Inspected"))        return false;    /* Items in player Bank slots, or transferred in/out - count: 2,066 */    if (!ProcessItemPacketsByOpcode(OP_ServerBankItems, "Bank"))        return false;    /* Items on merchant buyback list - count: 2,913 */    if (!ProcessItemPacketsByOpcode(OP_ServerMerchantBuybackItem, "Buyback"))        return false;

Ignoring the C++, just look at the /* Comments */ and the name of the opcode/category, and tell me what version of the item should be processed FIRST in order to give me the base (or what I'm calling Pure) item and it's base stats.

Just a quick explanation:
If I process "Server" items first (which I did in the last parsing), what Parser finds is all items that are sent to the players inventory when they log in. This includes all the augments, modifiers, crafting enhancements, etc that could be on an item that the player possesses. Let's say that's Item ID 1234.

Later in a different log, another player loots Item ID 1234 but of course there are no stats/modifiers on it yet because it's fresh loot off the ground. Unfortunately, because Item ID 1234 was already in the database, this new (pure) one gets discarded.

So I'm trying to fix that by processing the (pure) items first, and ending up with the (impure - or modified) items last so we still get them if they do not exist in a pure form.

Make sense?

Look over my opcode list order, tell me if that will result in the most accurate pure-to-impure parsing of items.

Whew. Thanks.

[John Adams]

Side question, that might change the above stuff (just thought about this heh)

When you link an item in chat to someone, or look up items with stats on a Merchant, do your skills/attributes affect that item? Or is it just when you put it on that your stats affect what is displayed on the item mouseover? By stats I simply mean the +STR +HP kind of stats, obviously not sockets or other augment methods.


Edit: Likewise, when you inspected another player, was the stats you saw THEIR stats, YOUR stats, or the base stats of the item (pure)?


Edit2: (hah sorry) Last one! For "Attachments", those Etherence, Primal things (sockets?) Did any items "off the ground" come pre-socketed? By off the ground, I mean looted, gained through questing, bought on a merchant/exchange, etc... any way of acquiring an item - did it come pre-socketed.

Because I am seeing "Attachments" on items from these opcode categories:

• Acquired
  Bought
  Chest
  Exchange
  Link
  Loot
  Market
  Merchant
  Reward

  [/list:u]
  
  Thanks!

[Jakkal]

Some augments already had sockets available (A lot of the raid loot did, for example.) Look up.."Socketed Ulvari Life Gem". That one came pre-socketed. No augment with a socket had an item in it already. Infact there was a bug in game that would not let you trade/sell/give an augment to another player if it had an augment in its socket.

When you linked stuff in chat, I cannot think of any reason why the stats on it would change EXCEPT, in Live Vanguard, if you inspected my toon and looked at my dagger, you would see this:
http://jakkal.com/vanguard/JhaaruBlade2.jpg

If I linked my dagger to you in chat, you would not see any of the purple text stats or ANY of the augments in the attachment slots.

That's the only example I can think of where if I link something to you, you don't see the same stats that I do.

I believe that the only thing that changed if you inspected another toon, or looked at linked gear was the ( % vs Lvl YOURLEVELHERE) on the items.

[Xinux]

I would go in this order.

OP_ServerLootList
OP_ServerAcquireItem
OP_ServerQuestRewardItem
OP_ServerHarvestingPotentialYield
OP_ServerMerchantItemMouseover
OP_ServerMarketItem
OP_ServerGrafts

*past this point i tried to list from least to most chance of a modified item*

OP_ServerBoughtTradeItem
OP_ServerMerchantBuybackItem
OP_ServerExchangeMouseoverItem
OP_ItemLink
OP_ServerAssemblyItems
OP_ServerBankItems
OP_ServerChestItems
OP_ServerItems
OP_ServerInspectItems

[John Adams]

Okay, Jak. We might be getting somewhere. So looking at the exact same item on 2 different characters at opposite ends of the level range, what I needed to see was that the "stats" didn't change.

level1.jpg

level50.jpg

What's changing is the % vs Lvl like you mentioned. That makes sense. Also notice, all 5 "Slots" of this item are [Empty]

This is the second part of my question - not whether items had "sockets" (sorry, I used the wrong terminology). I meant that items sockets were already pre-populated. Example from our new dataset:

pre-loaded-sockets.jpg

See how this BP has all 3 "Slots" already with something shoved in them?
Did anything in game drop off a mob this way? Or get looted >in any fashion< with slots already filled?

Because if not, then I should just delete the Attachments data entirely.

[Jakkal]

No, there was nothing in game that already had something inside an attachment slot. They *always* came empty.

[John Adams]

Good, thanks Jak for helping out. We devs have decided we're dropping all our parsed Slot data (it's really player data, thus dynamic) and with them, dropping any item "stat" that is Purple (modifier1, as it's currently called). A new items list will hit Targ within the hour, and I'll need devs and content members checking it out asap.

Basically hit an Exchange Broker and just look up all fancy items with stats and make sure they look "normal". There should be no more exploded values for stats (thanks Ratief).

[John Adams]

Oops. I think I just remembered why the Attachments (slots) table couldn't be deleted last year when we first discussed this. With no slot records parsed or loading at all:

Oops!.jpg

I am assuming this is incorrect?

[Jakkal]

...Nope.

[John Adams]

k, how about now?

fixed!.jpg

Where's my damn gold star?

Edit: Here's the comparison between ours and Xanadu

vg_item_1366.png

Zaseh.jpg