Important Dates

Currently Online

Latest Posts

Topic: Virtual Filesystem, Gamefile archive

Frostregen
Joined: 2009-03-02, 16:23
Posts: 2
Ranking
Just found this site
Posted at: 2009-11-21, 12:31

Hi,

how about some sort of virtual filesystem for widelands which would support stuffing all 20.000 single files into a single (or maybe a small number) of archives?

Motivation:

a) Currently the 20k files need about 1.26 GiB on my harddrive (64KiB Clustersize). Compared to 107 MiB worth of files data, only 12% space is actually used.

b) If done properly this would for sure improve map loading times.

c) The archive could transparently support compression, if wanted.

d) No problems with installing a new widelands version over an old one, as the whole gamefile archive would be replaced. (No old leftovers)

What do you think about this? (I'm willing to add this feature myself)

Greetings, Frostregen

Edited: 2009-11-21, 12:32

Top Quote
sigra
Joined: 2009-03-05, 19:02
Posts: 130
Ranking
At home in WL-forums
Location: Orsa
Posted at: 2009-11-21, 13:03

Frostregen wrote: how about some sort of virtual filesystem for widelands which would support stuffing all 20.000 single files into a single (or maybe a small number) of archives?

We are interested in PhysicsFS. But of course it must still work to have the files unpacked, especially for subversion checkouts. Even more interesting is to replace numbered sequences of PNG files with single MNG files.

Motivation: a) Currently the 20k files need about 1.26 GiB on my harddrive (64KiB Clustersize).

64KiB block size is considred an extreme case. Only use that for partitions with for example video archives. Normal programs often have many small data/configuration files. Home directories often also have many small files. So why not put things like /etc /home and /usr on partitions with small cluster size and tail packing?

b) If done properly this would for sure improve map loading times.

At least make sure to have latest SDL_image. A serious performance problem was fixed in version 1.2.8.

Edited: 2009-11-21, 13:12

Top Quote
Frostregen
Joined: 2009-03-02, 16:23
Posts: 2
Ranking
Just found this site
Posted at: 2009-11-22, 00:31

Nice to know it is already on the roadmap :) A simple config setting should allow to enable some "look on real path first, then in archive" method. About MNG/APNG... I don't know... Both seem quite unsupported in most tools. (Last time I tried them. ~beginning 2009) Personally I would use the good old sprite sheets face-wink.png

Clustersize: I know. I use 64KiB block size for my games partition only. (Ok, storage partition too) Since nearly all games use some sort of gamefile archive, this is/was not a problem, but helped. (Out of 63 games above 200MiB, widelands is one of three below 90% block usage efficiency) My Programs/UserData surely is on a different partition with standard 4KiB face-wink.png Does NTFS support Tail-packing? face-wink.png

Loading times are not a problem on my current machine. It's just quite obvious it could be faster.

All in all it is not really a problem, just something which could/should be done better. So I'll wait for PhysicsFS face-smile.png

-Frostregen


Top Quote