Page 2 of 3
Re: World Crash
Posted: Sun Sep 28, 2014 1:38 am
by Volt
[quote="Lokked"]I've committed Rev 676 to Dev SVN to attempt to mitigate the crashes in the OP, based on my post above.[/quote]
Cool Lokked!
We will need to commit a change to the vgo_world.sql as well (maybe I wasn't very clear above, but .rift to Isle of Shadow is what triggered this crash). Xinux posted above what needs to be done. Think we need John's assistance to adjust the vgo_world.sql.
Rev 701 - World Crash in Client::GetCharacter()
Posted: Fri Oct 10, 2014 1:00 pm
by John Adams
Stack:
Code: Select all
> WorldServer.exe!std::_Ptr_base<WorldCharacter>::_Reset<WorldCharacter>(const std::_Ptr_base<WorldCharacter> & _Other, bool _Throw) Line 363 C++
WorldServer.exe!std::shared_ptr<WorldCharacter>::shared_ptr<WorldCharacter><WorldCharacter>(const std::weak_ptr<WorldCharacter> & _Other, bool _Throw) Line 552 C++
WorldServer.exe!std::weak_ptr<WorldCharacter>::lock() Line 1051 C++
WorldServer.exe!Client::GetCharacter() Line 103 C++
WorldServer.exe!ChunkServer::DisconnectChunkClient(std::shared_ptr<Client> chunk_client, unsigned short reason_id) Line 2478 C++
WorldServer.exe!UDPServer::KickDupeClients(unsigned int account_id, unsigned int connection_id) Line 250 C++
WorldServer.exe!Net::HandleCharacterSelected(std::shared_ptr<Client> & client, PacketStruct * packet_struct) Line 377 C++
WorldServer.exe!Net::Process() Line 159 C++
WorldServer.exe!main(int argc, char * * argv) Line 187 C++
WorldServer.exe!__tmainCRTStartup() Line 241 C
WorldServer.exe!mainCRTStartup() Line 164 C
kernel32.dll!76a1338a() Unknown
[Frames below may be incorrect and/or missing, no symbols loaded for kernel32.dll]
ntdll.dll!77c49f72() Unknown
ntdll.dll!77c49f45() Unknown
Blackstorm and I were chunking around all over the world, dozens of chunks. I logged off a while ago and he continued to change chunks. This was him going into Khal, instant crash. Will try to reproduce.
[The extension log has been deactivated and can no longer be displayed.]
Little more info. I walked the stack to GetCharacter() and the character value is null.
Code: Select all
shared_ptr<WorldCharacter> Client::GetCharacter() {
return character.lock();
}
I also saw this whiz by the console this time:
[quote]14:11:11.943 W Chunk Camp requested from client with character==null
14:11:14.876 W Chunk Camp requested from client with character==null[/quote]
Says 2 Clients, 3 Chunks loaded though again I am sure it's only Blackstorm online.
[The extension log has been deactivated and can no longer be displayed.]
Edit2: Note no JOIN:
[quote]14:09:07.317 D Account Client Created: 3
14:09:07.317 I UDP New client connected from 87.231.45.133:51934
14:09:07.317 I UDP Received session request from 87.231.45.133:51934 with connection ID 245526177
14:09:07.598 I Chunk New Session: Account ID: 266 Session ID: 'r82iqc8l94or7iqe9dt5ulu7s7'
14:09:07.895 D Chunk control_text='HELLO REVISION=0 MINVER=3151 VER=3186'
14:09:08.144 D Chunk control_text='LOGIN'
14:09:12.762 I UDP Client from 87.231.45.133:52132 has been removed.
14:09:12.762 D Account Client Destroyed: 2
14:09:12.793 I Chunk Starting chunk shutdown timer for Qa Riverbank.
14:09:30.655 I Chunk Shutdown timer expired. Shutting down New Targonor.
14:10:18.095 D Char Saved character 'Black Crow' (451) data.
14:11:11.943 W Chunk Camp requested from client with character==null
14:11:14.876 W Chunk Camp requested from client with character==null
[/quote]
Re: Rev 701 - World Crash in Client::GetCharacter()
Posted: Fri Oct 10, 2014 9:56 pm
by Lokked
This bug has been with us for as long as I can remember!
Blackstorm, had your client crashed or shutdown when you first entered Khal? And then you logged back in, immediately? I ask because I see the .rift to Khal, and then I see the multiple character created/destroyed log lines that occur when you enter character select.
I have noticed that when the client crashes and a person logs back in immediately, it crashes the server every time with this exact error, during the KickDupeClients function.
If you can confirm exactly what happened during these points, we may be able to squash this once and for all.
Re: Rev 701 - World Crash in Client::GetCharacter()
Posted: Sat Oct 11, 2014 12:00 am
by Blackstorm
Hi,
Looked : "I have noticed that when the client crashes and a person logs back in immediately, it crashes the server every time with this exact error, during the KickDupeClients function."
True the second times. (the player controller freeze -> i can't move my character, only chat / escape button was available)
For the first time, i have a crash when i chunk to Khal after a long time (like said John) but i think (not really sure) he said "crash" at the same time i relog from this crash (at least, less than 2min after my first crash).
Should be due to the crash/relog..
Re: Rev 701 - World Crash in Client::GetCharacter()
Posted: Sat Oct 11, 2014 9:33 am
by John Adams
[quote="Lokked"]This bug has been with us for as long as I can remember![/quote]
Lokked is correct, I originally reported this same stack on Sep 25th. Merged these 2 posts into the original.
Re: Rev 701 - World Crash in Client::GetCharacter()
Posted: Sat Oct 11, 2014 10:06 am
by Lokked
[quote="Blackstorm"]Hi,
Looked : "I have noticed that when the client crashes and a person logs back in immediately, it crashes the server every time with this exact error, during the KickDupeClients function."
True the second times. (the player controller freeze -> i can't move my character, only chat / escape button was available)[/quote]
Now I'm concerned: Are you saying that this MOST RECENT session, while running around with John, you rifted/chunked and after loading you could not move your avatar? John, were you on Rev 700 or later? This was the whole issue Xinux and I were fixing and we probably chunked more than 500 times combined in testing and thought we'd eradicated this issue.
Re: World Crash in Client::GetCharacter()
Posted: Sat Oct 11, 2014 11:12 am
by John Adams
This was rev 701. Xinux committed unrelated changes at 10:46am AZ time, and the issues above were reported at 11:07 and 14:00 respectively.
For clarity, two logs posted above are 2 separate sessions. Both ending in a World crash of a similar nature (GetCharacter() with the character object being null). I was not present for either one. I was there initially in the first log merely to test the new code/db updates, then Exited the client. The first log should show my departure. The second log was all Blackstorm (pretty sure he was alone on the server at the time).
Re: World Crash in Client::GetCharacter()
Posted: Sun Oct 12, 2014 9:20 am
by John Adams
A slight variation on the current theme; Another crash, this time in GetClient(), but following the KickDupeClients() function like the first crash mentioned in this thread.
Stack:
Code: Select all
ntdll.dll!77cb8277() Unknown
[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]
WorldServer.exe!Mutex::ReadLock() Line 99 C++
> WorldServer.exe!UDPServer::GetClient(unsigned int account_id) Line 188 C++
WorldServer.exe!ChunkServer::GetClient(unsigned int account_id) Line 274 C++
WorldServer.exe!UDPServer::KickDupeClients(unsigned int account_id, unsigned int connection_id) Line 252 C++
WorldServer.exe!Net::HandleCharacterSelected(std::shared_ptr<Client> & client, PacketStruct * packet_struct) Line 377 C++
WorldServer.exe!Net::Process() Line 159 C++
WorldServer.exe!main(int argc, char * * argv) Line 187 C++
WorldServer.exe!__tmainCRTStartup() Line 241 C
WorldServer.exe!mainCRTStartup() Line 164 C
kernel32.dll!76a1338a() Unknown
ntdll.dll!77c49f72() Unknown
ntdll.dll!77c49f45() Unknown
Code:
Code: Select all
shared_ptr<Client> UDPServer::GetClient(uint32_t account_id) { shared_ptr<Client> client; m_clients.ReadLock(); for (auto& itr : clients) { //<<== here, clients = empty if (itr.second->GetAccountID() == account_id && itr.second->IsConnected()) { client = itr.second; break; } } m_clients.ReadUnlock(); return client;}
Console log was just someone looking up their characters:
[The extension log has been deactivated and can no longer be displayed.]
Re: World Crash in Client::GetCharacter()
Posted: Sun Oct 12, 2014 9:44 am
by Lokked
Yep, same thing: they pressed the play button at char select and kaboom.
I think for interim, I'll put a try catch around that function and just drop them if it errors. At least the server will survive if this happens.
Re: World Crash in Client::GetCharacter()
Posted: Sun Oct 12, 2014 10:19 am
by John Adams
Any idea why this would suddenly surface? We've been running a relatively stable server for months with many people crashing and coming back within 30s (damn that efficient loading VGClient:)) and it's never crashed the server before. A catch will work, but I'm wary of too many bandaids being applied, being a root-cause kinda guy.
It's not critical enough to hold the Public SVN push, because it does seem infrequent, and I believe you know what causes it so we can just warn the public "don't reconnect too fast". Likely only a New Telon problem regardless. When you (or other devs) get time, let's find a true fix, and I'll take your try/catch in the interim (so other clients are not effected by the kaboom).
Thanks Lokked.