(UPDATED 12:08 PM 11/11/09)Runic Patch 1.12 - Vendor updates

Community support forum for the Torchlight single-player game.

Re: Current Bugs under Investigation and Status

Postby Citizen86 » Wed Oct 28, 2009 4:54 pm

travisbaldree wrote:
Seems that the game simply reads the data directly from the zip, as if it were a folder now, as opposed to uploading it all into RAM and virtual-memory, so it can translate it into a series of data that windows file-system can comprehend. I also see less memory being used by the game, and the frame-rate seems more constant for me. (Went from 23.5FPS to 42.5FPS average, except for the skeleton-delay when smashing containers now. All seems fine. Little to no regular interval lag.)


You are awesome. Thanks for the investigation!
We found TOTALLY DIFFERENT performance issues with resource groups and the way they were being used, and a load of extra 'createfile' hits that were being spammed on a resource search, which also provided sizable speed improvements.
We'll investigate this with our internal build system, and if we're lucky, the wonder twin powers of both solutions can activate for a very good speed win.

Thanks again!

Travis


I look forward to getting more performance and quicker loads with your patch as well :D
User avatar
Citizen86
 
Posts: 51
Joined: Sat Oct 24, 2009 5:46 am

Re: Current Bugs under Investigation and Status

Postby Blitz » Wed Oct 28, 2009 5:15 pm

ISAWHIM wrote:Solution:
- Go to the programs folder. ("C:\Program Files\Torchlight")
- You need to unzip the "Pak.zip" contents. (Windows will unzip to a folder called "Pak", inside the game folder.)
- Now, you have to enter the "Pak" folder, and "Right-Mouse Click" the "media" FOLDER. Select "SEND TO" and then "Compressed (Zipped) Folder".
-This creates a ZIP FILE called media.zip, which has the folder and the contents of the "media" FOLDER inside the zipped file.
- ReName this "media.zip" to "pak.zip".
- Now select that file with the "Right-Mouse button", and then select "CUT". or (CTRL+X)
- Go back into the "Torchlight" folder and rename the original "Pak.zip" to the name, "OriginalPak.zip".
- Now you can PASTE the copied file into this folder. or (CTRL+V)
- You can delete the uncompressed folder once you have moved the zip file into the location of the original zip file.


I followed these instructions and the load time feels the same. Still 1 minute to get into main menu screen. 1 minute to get into the game. I do see the new zip file is half the size.
Blitz
 
Posts: 6
Joined: Tue Oct 27, 2009 7:32 pm

Re: Current Bugs under Investigation and Status

Postby DresdenDoll » Wed Oct 28, 2009 5:32 pm

Posting to confirm - this method cut my load times in half.
In the wake, our existence. In our parades... and in our dances.
With great spirits proclaiming that capitalism is indeed organized crime - and we're all the victims.

Image
User avatar
DresdenDoll
 
Posts: 167
Joined: Fri Oct 02, 2009 9:27 pm
Location: Louisville, Ky

Re: Current Bugs under Investigation and Status

Postby bentosta » Wed Oct 28, 2009 6:23 pm

Etrust antivirus also seems to be throwing the same error:

The Win32/Unknown was detected in C:\PROGRAM FILES\RUNIC GAMES\TORCHLIGHT\TORCHLIGHT.EXE.
....
File Status: No cure for this infection.
bentosta
 
Posts: 1
Joined: Wed Oct 28, 2009 6:21 pm

Re: Current Bugs under Investigation and Status

Postby yukonerdave » Wed Oct 28, 2009 7:15 pm

Just did ISAWHIM's fix and it made a HUGE difference, both in load times and in general gameplay. Thanks man! I did not official tests but I'd say it is at least twice, maybe 3 or 4x faster.
yukonerdave
 
Posts: 2
Joined: Wed Oct 28, 2009 7:13 pm

Re: Current Bugs under Investigation and Status

Postby Inverness » Wed Oct 28, 2009 8:41 pm

ISAWHIM wrote:NOTES: The compressed zip file is half the size of the other PAK file. This also seems easier for windows to read into memory and extract, as this is windows native compression codex for ZIP. I suspect the other ZIP is compatible, but not a native windows ZIP compression. All ZIP files are not the same. They make zips that are simply compatible for extraction, but they are not always fast, or good. I would guess they are using a non-windows zipping tool. They should be using windows native internal ZIP functions. (Placing anything into a compressed zip-folder makes it zipped. A nice feature since Windows 98se, that few people even realized was possible. No-one ever needed win-zip, it was a windows core function.)

Seems that the game simply reads the data directly from the zip, as if it were a folder now, as opposed to uploading it all into RAM and virtual-memory, so it can translate it into a series of data that windows file-system can comprehend. I also see less memory being used by the game, and the frame-rate seems more constant for me. (Went from 23.5FPS to 42.5FPS average, except for the skeleton-delay when smashing containers now. All seems fine. Little to no regular interval lag.)

The problem isn't Windows specific. The Windows created zip file using the Deflate method (algorithm) for files and the Store method for folders, this is normal zip file behavior. It is not using any windows-specific method. Now the original Pak.zip used Store for both files and folders. The Store method indicates that the data is being stored in the zip file as-is and is not compressed. The problem seems to be what I am mentioning below: when using the uncompressed Pak.zip, the program seems to read 30 times less data per call to ReadFile than it does with a compressed file.
travisbaldree wrote:
Seems that the game simply reads the data directly from the zip, as if it were a folder now, as opposed to uploading it all into RAM and virtual-memory, so it can translate it into a series of data that windows file-system can comprehend. I also see less memory being used by the game, and the frame-rate seems more constant for me. (Went from 23.5FPS to 42.5FPS average, except for the skeleton-delay when smashing containers now. All seems fine. Little to no regular interval lag.)


You are awesome. Thanks for the investigation!
We found TOTALLY DIFFERENT performance issues with resource groups and the way they were being used, and a load of extra 'createfile' hits that were being spammed on a resource search, which also provided sizable speed improvements.
We'll investigate this with our internal build system, and if we're lucky, the wonder twin powers of both solutions can activate for a very good speed win.

Thanks again!

Travis
I hope my post about the massive number of unnecessary CreateFile calls was able to help. I'd also like to know if you were familiar with the next post I made about the even greater number of ReadFile calls being made by the game, the vast majority of them to Pak.zip with read lengths in the range of 4 to 20 bytes. There were 2.7 million ReadFile calls by Torchlight.exe as a result of starting the game, selecting my character, loading it, then exiting the game after a second or two.

I wonder how much this has to do with the format of the zip file, so I'm going to test it again using the fix mentioned earlier.

Edit: WOW! Massive performance improvement! What was once 2.7 million ReadFile calls has been reduced to 85.7 thousand! A great deal more data is being read from the zip per call to ReadFile, roughly 30 times more! While there are still a lot of unnecessy CreateFile calls, it's the ReadFile that seems to be the major culprit of performance issues.

Note that I have a 3.16GHz Intel Core 2 Duo, so I imagine it is much easier for my CPU to read large chunks of compressed data and decompress it than it is to read tiny bits of uncompressed data.

Edit 2: Made another observation: once I had loaded into the map, the majority of ReadFile calls were to Town.ogg and with Title.ogg coming before and after that during the title screen. Procmon indicates that ReadFile returned END OF FILE several times while playing both of these music files. I would recommend loading the whole music file for the current map into memory to reduce disk I/O.

I hope that helps.
User avatar
Inverness
 
Posts: 158
Joined: Tue Oct 27, 2009 10:13 pm
Location: Houston, Texas

Re: Current Bugs under Investigation and Status

Postby travisbaldree » Wed Oct 28, 2009 10:15 pm

I hope my post about the massive number of unnecessary CreateFile calls was able to help. I'd also like to know if you were familiar with the next post I made about the even greater number of ReadFile calls being made by the game, the vast majority of them to Pak.zip with read lengths in the range of 4 to 20 bytes. There were 2.7 million ReadFile calls by Torchlight.exe as a result of starting the game, selecting my character, loading it, then exiting the game after a second or two.

It was an invaluable lead - thanks a lot!
You should find it massively reduced with the patch when we get it out the door-

On those oggs - those are actually streaming, so we expect to see them constantly buffering chunks of song as they go, then looping
travisbaldree
Runic Retired
 
Posts: 1612
Joined: Fri Jun 05, 2009 4:17 pm

Re: Current Bugs under Investigation and Status

Postby sos4tw » Wed Oct 28, 2009 10:16 pm

Does the update to your first post about the long loading times mean we can expect a patch early tomorrow? =)
World of Warcraft is the Devil.
User avatar
sos4tw
 
Posts: 74
Joined: Tue Oct 27, 2009 8:17 pm
Location: California

Re: Current Bugs under Investigation and Status

Postby Suzi » Wed Oct 28, 2009 10:29 pm

I want to report that the workaround did NOT work for me, my load times hardly quicker. Windows XP Pro SP3 still needs ages to open the zip if i double-click it. Same with a zip created with 7-zip.
Suzi
 
Posts: 3
Joined: Wed Oct 28, 2009 10:24 pm

Re: Current Bugs under Investigation and Status

Postby travisbaldree » Wed Oct 28, 2009 10:31 pm

Probably not! Folding the fullscreen fix in as well - and we're almost done with that.
travisbaldree
Runic Retired
 
Posts: 1612
Joined: Fri Jun 05, 2009 4:17 pm

Re: Current Bugs under Investigation and Status

Postby Inverness » Wed Oct 28, 2009 10:38 pm

I hope the bug fixing being done now hasn't pushed the release of the editor back. :cry:
Suzi wrote:I want to report that the workaround did NOT work for me, my load times hardly quicker. Windows XP Pro SP3 still needs ages to open the zip if i double-click it. Same with a zip created with 7-zip.
What level of compression and algorithm did you use to create the zip file? The fix instructions involved using the normal Windows compression which uses the Deflate algorithm with a normal level of compression. Specifying a higher level of compression or a different algorithm in 7-Zip will invalidate the performance gains.
Last edited by Inverness on Wed Oct 28, 2009 10:43 pm, edited 3 times in total.
User avatar
Inverness
 
Posts: 158
Joined: Tue Oct 27, 2009 10:13 pm
Location: Houston, Texas

Re: Current Bugs under Investigation and Status

Postby LtgCrabfish » Wed Oct 28, 2009 10:38 pm

Hey, Travis, is the crash looked on to?

Or would it happen to be this?

"H) Startup issues on some hardware/resolution combinations. We think we know what this is, and are probably going to refactor some of the startup fullscreen code in an attempt to take care of it. We may gather a few guinea pigs to help us out. ( UPDATE - In progress - going to call this 50% )"

Hopefully the crashes are getting fixed. Been trying to play this game for 2 days now lolol!

Thanks!
User avatar
LtgCrabfish
 
Posts: 199
Joined: Thu Oct 15, 2009 1:57 pm
Location: Toronto

Re: Current Bugs under Investigation and Status

Postby Hyfar » Wed Oct 28, 2009 10:58 pm

Not sure if this is the right place to ask, but when can we except the patch?
Hyfar
 
Posts: 26
Joined: Wed Oct 28, 2009 4:42 am

Re: Current Bugs under Investigation and Status

Postby Maurog » Wed Oct 28, 2009 11:11 pm

As a WinXP SP3 user, I can't wait to try out ISAWHIM's solution later today and see if it improves performance. But that's not what I wanted to say - I just want to tell all you Runic devs that you're doing a great job. Your policy of "acknowledge bugs and work with the users to fix them" is awesome, and a refreshing change from the "pretend nothing is wrong, don't comment on any complaints, fix the issue in a patch two months later or never, depending of how the coin flips" policy that many developers use.

Stay cool, guys. We know you have a lot on your plate right now, and trust you to fix things as quickly and efficiently as you can. Pay no heed to the squeaky wheels who denounce your game but have nothing constructive to offer.
Slay the living! Raise the dead! Paint the sky in crimson red!
User avatar
Maurog
 
Posts: 210
Joined: Mon Oct 19, 2009 10:11 am

Re: Current Bugs under Investigation and Status

Postby ThndrShk2k » Thu Oct 29, 2009 12:02 am

Hopefully soon. I would assume when they release TorchED myself, but could be sooner, or later. But most likely within a week at the most.
TL2 Dwarven Class Mod Discussion
Current Goal: Milestone 1 - Model Insertion
Status: Looking for Modelers and Animators.
ThndrShk2k
 
Posts: 130
Joined: Sat Oct 24, 2009 3:36 pm

Re: Current Bugs under Investigation and Status

Postby Oconnel » Thu Oct 29, 2009 12:16 am

You are awesome. Thanks for the investigation!


I want to point out, that ZIP fix does absolutly nothing on my machine. Loading times are still very long (Windows 7).

I have a better idea. Why don't you just unpack all that stuff, so there would be no compression at all. It seems reasonable?
User avatar
Oconnel
 
Posts: 98
Joined: Fri Jun 05, 2009 11:11 pm
Location: Poland

Re: Current Bugs under Investigation and Status

Postby travisbaldree » Thu Oct 29, 2009 1:45 am

I want to point out, that ZIP fix does absolutly nothing on my machine. Loading times are still very long (Windows 7).

I have a better idea. Why don't you just unpack all that stuff, so there would be no compression at all. It seems reasonable?


You'll actually probably get a big win from the OTHER half of the fix that comes with the patch.
In fact, unpacking them makes things WORSE - tons of file hits.
travisbaldree
Runic Retired
 
Posts: 1612
Joined: Fri Jun 05, 2009 4:17 pm

Re: Current Bugs under Investigation and Status

Postby travisbaldree » Thu Oct 29, 2009 1:46 am

Hopefully soon. I would assume when they release TorchED myself, but could be sooner, or later. But most likely within a week at the most.

We are aiming for less than a week -
travisbaldree
Runic Retired
 
Posts: 1612
Joined: Fri Jun 05, 2009 4:17 pm

Re: Current Bugs under Investigation and Status

Postby ISAWHIM » Thu Oct 29, 2009 4:36 am

C) Poor ingame performance on certain machines, when there really shouldn't be - this may be related to B) - under investigation ( UPDATE - some improvement due to B's fix as well! )


Is this about the random hesitation, where the game slips into 1 FPS for a minute?

I would like to add, it is exactly 160 frame-counts (Ticks), unless interrupted by a level-load or portal.

Other attributes:
- The mouse and sound play normally.
- The delay is exactly 1 second between frames or more. (Mostly 1 second.)
- The pet-timer confirms that the issue is inside the game-timer, not a graphic-card issue.
- CPU use jumps to 99.5%, but hard-drive access is not present. This suggests a dead-loop game-timer delay.

Debug and check to see if the game-frame-timer is somehow being lost or accidentally reset with an integer value of 1 or 0. I suspect that when this occurs, the game is checking to see if 0.01666 seconds have passed, and is holding the frame-count in a dead-loop to limit frame rates. (If the value is 0 or 1, the frame rate would be exactly one second, depending on the time comparison. The value would not be fixed until the end of the frame, but for 160 loops when the value is lost, it shows as 1 FPS. Because the comparison happens at the opposite side of the frame processing.)

This can happen with FLOAT if the value is not large enough, or the value is an extended notation value too small but not zero. 0.1E99 might be 0 or 1 as an integer if adjusted floats are used, where floats are limited to say... four places and rounded, not clipped. Values greater than 0.1 should be forced to be 0.1, which translates into 10 FPS, values lower than 0.005 should be forced to be 0.005, which is your 200 FPS upper range.

NOTE: I have set my max FPS to 30FPS now. No sense processing 200 FPS when most monitors only display 60 FPS. (60 Hz is 60 FPS, You get more screen tearing and drive CPU use up higher when you try to process 200 frames. At 100Hz, 200FPS translates into two half-screens being drawn. Read the HZ of the monitor and set the maximum FPS to match that. Unless you are benchmarking, which destroys hardware and consumes tons of wasted power, you don't need more frames than can possibly be seen by the human eye or displayed by the monitor.)

Just my 2-cents, not a demand... LOL

P.S. Good work, and I am glad that the zip thing helped both issues a little. I can now run full resolution with all effects turned on. But the random delay still exists. Now it is random, not every nine minutes, like before.
Looking for a break from the game? Waiting for more content?
Read my short story... [Story] Miles deep and miles away
Story by: Jason D'Angelo
User avatar
ISAWHIM
 
Posts: 224
Joined: Sat Oct 24, 2009 6:07 am
Location: Jacksonville, Florida

Re: Current Bugs under Investigation and Status

Postby Eversor » Thu Oct 29, 2009 4:51 am

Wonderful work guys! Amazing developer-customer interaction, incredible for this days.

What about this problem?-->> viewtopic.php?f=24&t=1627

Is there any hope for us, the cropped torchlight 1920x1200 gamers??? Pls? any word appreciable
Eversor
 
Posts: 10
Joined: Tue Oct 27, 2009 4:54 pm

PreviousNext

Return to Torchlight Win/Mac/Linux Support

Who is online

Users browsing this forum: No registered users and 2 guests