Use this thread for discussion on the raid lag. We have been investigating lag in general so any info is helpful.
Some things known to cause lag: Multiple add spawns Raid AoE'ing adds Deaths Faction updates Certain geometrical locations such as the roof above Sil'mr-Li (in theory) Burn phases Certain parts of some fights - ie IWG at 30%, Karax during the light dance
There still seems to be an issue with VG and management of video RAM, which probably isn't helping. It seems that vram usage climbs over time, even if you're not really moving anywhere to cause new textures etc to be needed. Not really sure how to be more speceific, but even on low textures / high performance VG has been using the full available vram (I've tested this on a 6600GT, 8800GT, GTX460 and GTX660 Ti) and then degrading performance once it starts inefficiently paging data in and out. /flush temporarily resolves it, but is a bit of a pain.
Especially deaths are an annoying source of raid lag because it spirals out of control quickly. One or two DPSers die, which would be no major issue, but the lag caused by their deaths makes healers miss their time by a second or two.. leading to more deaths.. and within a short time you've got 5 seconds of frozen picture, and when it's gone, the wipe is done. The recent changes done to the memory / texture loading mechanics weren't a real improvement either. Before the last patch I could run around smoothly, and objects with missing textures would pop or fade into view over a few seconds. Sure, this caused seconds of transparent houses etc, but I could _move_. Now when an object with new textures/animations/sounds gets into view (like someone running past me, or starting to fight in my vicinity etc), the client freezes for a second or two until the data is loaded completely. This is at least as bad as lag in raids.. because once an add attacks the group that you haven't fought before, the client freezes to load the add's combat animations and sounds. Would help a lot if that "improvement" could be reverted. I don't mind not hearing a mob's first attack, but I'd like to be able to keep on moving. On second thought.. could the death lag actually be related to this? Clients freezing to load other people's dying animations?
To be honest i think a very large part of all the hitching and supposed lag is simply due to a generally poor coding scheme, where the client is blocking due to either loading from the harddrive or waiting for a server message. Switching all of this over to an asynchronous style however is an immense task for the developers
Forgive me if I have missed this before but, this is the first I have heard about lag related specifically to death. When did this start?
Memnoch wrote: At least the harddrive part worked fine until the patch ~2-3 weeks ago.. as I wrote already. You had missing walls and objects which faded into view one by one, and sometimes combat sounds were delayed by a second or two, but the client moved on smoothly while catching up with loading content. Now they've flipped a switch somewhere and the client is freezing while loading, with all consequences this brings (like lag-running off cliffs etc..), all to make the client work on lesser equipped PCs, as one of the Devs stated. While the client is freezed for loading, the server will mostly think you're still running.. which means that your pets, people in /follow etc.. will continue running in the direction you were going before the freeze.
Silius wrote: Hmm, im sure ive mentioned it a few times before but ill give some details If i die in a 24 man encounter, i usually get a 2-3 second game freeze before i see myself dead, its simular to the pinglag reported before. except its always been present since the start of vg raids. Sometimes, more often that not, a single person dying and releasing can cause others to expirience ping / fps issues. i hvent been able to pin it down to either dying or releasing though. leaning towards the release. (this is since april patches) If lets say a raid wipes entirely at once, due to some deathblast, like teraxis, slimir, jerkj etcs. The same freeze happends to everyone, and is extremely long, up to 10 seconds. if a person were to survive and 23 ppl die he would lag out as well. You dont actually have to see the encounter to get the lag, ive been crafting 200m away for jerkj, while 3 groups wiped on the iceblast. i still got the freeze. I believe the effect occurs after the death damage is taken, and the client is perhaps struggling to process it. There is also 1-3 seconds lag if someone rezzes you, any time you get rezzed there is some lag, more in an active raid encounter. ( ps : there are issues with people not getting rez windows if not released /rez doesnt work either, higly annoying with cross chunk ressing broken). (maybe u fixed this already, dunno, cant play from here) There is lag in general with big (giant) encounters, like arachnidon, karax, fengrot. There is an item that lets you make the mob smaller, increases you fps by a ton. we think this is due to the size of the particles, both of the giants abilities and our abilities with effects done on them. (also large encounters are hated cause you cant see the damage numbers above their heads, and u cant zoom out far enough) Hope this helps, Alimora
I think the single biggest source of raid lag is from stats being completely re-calculated every time a single buff or debuff is added or removed. This problem is further compounded primarily by % damage (de)buffs, but also any other (de)buff that modifies a percentage of the total.
Silius wrote: It's done it for.. as long as I remember, though it seems to be worse now than previously. I believe it's partly due to the UI updates - and lets face it, the Raid UI has always been a source of issues, with group member lists bugging out, alive/dead, distance etc all causing problems, especially when zone changes are involved - but also partly due to redrawing the name (I find it less noticable with name displays off) and presumably the death animation / model change / redraw as tombstone etc if they release. But also, if the person dying had an aura-type effect up, it seems to be more pronounced - possibly due to recalculating the impact on this change in buff level across a larger number of characters. But, yes, when anyone dies, it seems to cause a brief pause - in the past some of us have used it as an indicator that someone died, even if it's otherwise hard to tell through all the clutter Whether it's coincidence or not, our raid last night was probably the best performance in terms of ping stability and framerate I've had in a raid in a long time - until my DSL dropped out, but that one isn't VG's fault We did the same mobs as last time, yet when I got 0 - 5fps on Whitewind last week, I rarely dipped below 10fps last night. And my ping, when it did spike, was only to ~1200ish (previously has been spiking to 3 - 4k, sometimes 9999).
Khabarakh@Seradon wrote: Just wanted to confirm this. It's definitely worse now than in the past, but has always been present. I have to disagree with it being tied to release, unless you are talking about the rest of the raid getting the lag (position update maybe). The best way to see this is to get in a 24 man raid and we'll let you get killed. You will have a good 2-8 sec complete freeze while the server deals with the math. 2 sec if you're the only one dying, and it gets exponentially worse if you have multiple deaths at once. From my perspective, this has definitley gotten worse lately. X9100 - 3.4 ghz 8 gigs of RAM VG and OS only on an SSD 9800M GTS 1gb video Bonded pair DSL - 10 mbps down, 2 meg up
Fogerty wrote: Emphasizing what Fogerty said, whenever I've gotten this lag spike, that's how I know I've died. Before the changes in April, whenever I had this kind of lag, I knew my character was dead. But now I'm uncertain if I've died or if we're getting lag spikes from the server.
Raid Lag has always existed and is sourced by many events. There is not one simple source of the issue. Back years ago when raiding with BotS, it was always an issue. Certain raids would lag out and eventually kick most raid members. So it sounds like nothing has changed in that regard. I always had much better success at reducing my lag and staying connected where others dropped, because I was always ahead of the curve with technology. I live in California (thus close to Sony servers in SD), and have always enjoyed much higher internet connection speeds than most using ISDN and Cable. My connection is usually in the range of +20meg Up/down via a home fiber connection. One of the key factors being high Upload speed. Many user might have a high speed download connection these days, but upload is very slow still. Before SSDs I created a RAID 0 of two 10K velociraptor drives for my gaming. This was very $$$ to do four years ago, but extremely fast, faster even today then some early SSDs. Once SSDs appeared I bought and used them early on, and saw large improvement in chunking and loading, almost to the point it was seamless in some areas. I have 1gen, 2ndgen and now I just added a new 3rd gen 90Gig SSD. Also many if not all raiding done players use Vent, which eats up upload bandwidth. And since most players do not have high upload speeds it may also play into the issue. The raid lag, and probably the entire lag issue is complicated by many factors. Designers and programmers are not necessarily going to have the skills to troubleshoot this. I assume there are some engineers involved, if not there needs to be. Get some systems/network engineers(that knows their stuff) into an actual raid so they can experience the issues and diagnose the probable causes. I work as systems engineer for a very large gaming company, and we are constantly debugging issues created by bad code pushes, and design conflicts. Sometimes it is very complex and difficult to troubleshoot, sometimes the fixes are very simple and emberrassing. I have to play the games I support to keep my finger on stuff like lag and poor performance. When I hear a game designer/developer says, what lag? Lag in raids? I know they are not playing the game. A bad sign. I am still shocked that these issues continue to plague VG. I am close to assuming it is so firmly built into the code that nobody is going to resolve it. If the lag issue could be solved once and for all, there is really no limit to how popular VG could get. Get the engineers and get it solved or VG will begin to die another slow death.
Zewtastic wrote: Yeah, a lot of players with bleeding edge systems and connections. They're still going to experience a lot of the network lag and death pauses like the rest of us. It's an ongoing battle. Hopefully we can get the stability we're looking for in the future. Like you're explaining though, it may not be tenable. Although, the very fact that it got worse, tells me that it can get better. If you're a glass half full kind of guy. The fact that Sil never got wind of it 'til now just tells me that Devs/GM's rarely, if ever, actually die while they're playing. Especially not in a raid environment that's doing it's best to task the bandwidth.
we have been told that none of the devs have actually played lvl 55 characters in real raids, so we invite them to our raids with gm style characters. they are not active participants, nor are they listening to our vent chat. So really all they can see is the connection meter and a few other tools, they likely are not getting the first hand connection between lag and events in raid. It would make sense for some of them to be included in the raid itself and then buff, cast, die, etc like the rest of us. It may help. We had a successful raid tonight with several useful kills for newer players. Every one of them was dogged by lag spikes and loss of buffs to chunk crashes (khal, isle of garg). All we can do as players is keep feeding the best information we can, and present our theories. somewhere in all this may be useful information to help solve the problems.
Last week we killed IWG with some lag spikes throughout the fight and a 5 or so second lag spike at 30%. When we came out of this lag spike, everyone was still alive and nothing had registered from either us nor IWG. Hence, we could complete the fight as per normal. The encounter worked, albeit with difficult lag. Tonight, after the changes to IWG (which I stongly protested), the lag spikes throughout the first 70% were variable and noticeable, but again nothing was registered during those lag spikes, so that no one died while we were lagging out. However, as I feared and predicted in the previous posts, Silius has changed a working fight and now made it unkillable at 30%. It is not unkillable due to the increased difficulty in the mechanics (although the fight would now be MUCH more difficult if it was possible), but because now when you hit 30%, regardless of the number of buffs on characters before that mark, the game completely freezes for well over 10 seconds during which time IWG continues to kill people while not registering new input from users. In all our attempts, shortly after 30%, we would lag for an extended time over 10 seconds, return to find 3/4 dead people, freeze for another 2 seconds, and return to find 2/3 more dead people. This continued as the new abiilty attempted to strip all buffs repeatedly. Net scoop, the 4th hardest mob in game has been screwed up and is now MUCH laggier due to the changes, in addition to being much harder even without the lag. Well this is hardly a suprise for those that know your work, I would ask that the mob be returned to a working encounter until lag is fixed. At that point, if you want to try and make the mob much harder so that only 0, 1 or 2 guilds can kill it, I would welcome the challenge, but even then, it would be a poor decision to remove content from any but the top guilds in a struggling game.
Naghite wrote: I agree 100% with this because I was there. Would like to add this to the last part of the post: Why screw with working content and even more why remove it. I thought the object was to add "WORKING" content not remove it. You can make it killable by putting it back the way it was and you can make it have less lag to move him to 1 of "MANY MANY" empty Kojani chunks. Kinda like you did when you tested it last week. Which brings me to this. Last week when you tested it for the new mechanic why would you not test it where it is now why move it to a completely different chunk ? Oh I know the "LAG" was to bad for you even to test it where its at so then why the heck put it back there? Always forward never backwards.....................
Last week the changes to IWG were already there so you fought the same mob twice and got different results?