Tool to download cache files

[chevket]

Hello,

I have developped a perl script to automatically download cache files.
It's the first release and some functionnality doesn't work yet.
Find it here :
http://chevket.turim.free.fr/divers/WorldWind/DownTiles.zip

Limitations :
- you can only download JPL Global Landsat Mosaic (Visible Bands - Panshapened), if you change it, it may corrupt the tile-size.
- the dem files are also downloaded but only in png format (I need to find a tool to transform it to Tiff)
- there is a command line windows which appear, I don't have find yet the solution to hide this. I use s3tc.exe (with its dll vic32.dll) to transform the jpg files to dds file because I don't have find how to get the same result with nvdxt.exe (I lost colors with nvdxt, if someone now the correct parameters ?)
- maybe some bugs with differents starting and ending Lat or Long ?

Things to do :
- find the urls to download the others layouts (informations are welcome)
- put all the config information in a config file to allow people to add thiers own config
- hide this annoying windows - Done
- add some logs to know what we have already download
- add an option to download all levels from one level to an other.

I will post a new version when all this "to do" point will be Done. But before, I'm waiting the new version...

[meegja]

Originally posted by chevket@Feb 13 2005, 04:20 AM
Hello,

I have developped a perl script to automatically download cache files.

Quoted post


Sounds cool! But on the other hand, wouldn't a tool like this put more stress on the JPL servers?

[canosso]

Hello chevket,
there are a few things which are against using your script at the moment:
1.) We are on the eve of World Wind 1.3 which should have a new, more detailed mosaic. This mosaic should be downloaded from a different server, not from JPL and it would have an other URI.
2.) The size of the downloaded DDS file, the downloaded files by World Wind have the size of 129 KB, the converted by script 171 KB which makes the cache a third larger than if downloaded with World Wind.
3.) At the moment there are no Terrain files, without 3D World Wind has less fun and use.
4.) World Wind 1.3 should bring faster converting of the tiles.

canosso

[chevket]

Originally posted by canosso@Feb 13 2005, 01:25 PM
Hello chevket,
there are a few things which are against using your script at the moment:
1.) We are on the eve of World Wind 1.3 which should have a new, more detailed mosaic. This mosaic should be downloaded from a different server, not from JPL and it would have an other URI.
2.) The size of the downloaded DDS file, the downloaded files by World Wind have the size of 129 KB, the converted by script 171 KB which makes the cache a third larger than if downloaded with World Wind.
3.) At the moment there are no Terrain files, without 3D World Wind has less fun and use.
4.) World Wind 1.3 should bring faster converting of the tiles.

canosso
Quoted post



1- actually the URL is harcoding in the exe file, but my goal is to define it in a text file, the differents options as source name file, number of levels and sizetile will be store in the texte file, so we may with the next version add sources to the tool.

2- my dds files are 129Kb file as the files from World Wind. World Wind download jpg files and convert it to dds files, so I shouldn't lose more data than World Wind by recompress the data.

3- my tool download Terrain files as World Wind do it, the only problem is that my tool doesn't convert this files from png to tiff files as World Wind do. If you have a mass converter you can convert all the files by yourself on your computer until I find a solution.

4- yes, I don't know what will be able to do World Wind 1.3, If there is an automatic script inside which is able to download region without help of the user and in a good way, not like with the javascript from the other post, my tool will be unusual.


About the question :
But on the other hand, wouldn't a tool like this put more stress on the JPL servers?

No, my tool only do simple thread download, not multithread (I don't know how to do multithread with perl and it's not the goal), so it doesn't stress server more the World Wind... if you don't try to download all the Earth... But as only a stupid guy will try to download all the Earth... And actually, you can only download a level, so you can't download all the Earth in one shoot...



About this question of data download, why World Wind doesn't use directly jpg files from the server instead of compressing it again in dds files ??
We lose data information if we transform the files !

[canosso]

Hello chevket,
Originally posted by chevket@Feb 13 2005, 05:25 AM
3- my tool download Terrain files as World Wind do it, the only problem is that my tool doesn't convert this files from png to tiff files as World Wind do. If you have a mass converter you can convert all the files by yourself on your computer until I find a solution.

4- yes, I don't know what will be able to do World Wind 1.3, If there is an automatic script inside which is able to download region without help of the user and in a good way, not like with the javascript from the other post, my tool will be unusual.
Quoted post


Did you try the two Gdal files in the Utils folder, they should make terrain files. For Information about the GDAL look http://remotesensing.org/gdal/gdal_utiliti...#gdal_translate (http://remotesensing.org/gdal/gdal_utilities.html#gdal_translate).

At 4. I changed the script to the moviescriptprompt.txt, there the user has only to input the two different Latitiudes and Longitudes and select the Altitude, Delay, Difference, and all higher levels will be downloaded automaticly, not the script is the problem, World Wind has high CPU usage and therefore everybody has to wait so much.

canosso

[chevket]

Originally posted by canosso@Feb 13 2005, 04:01 PM
Hello chevket,
Did you try the two Gdal files in the Utils folder, they should make terrain files. For Information about the GDAL look http://remotesensing.org/gdal/gdal_utiliti...#gdal_translate (http://remotesensing.org/gdal/gdal_utilities.html#gdal_translate).

At 4. I changed the script to the moviescriptprompt.txt, there the user has only to input the two different Latitiudes and Longitudes and select the Altitude, Delay, Difference, and all higher levels will be downloaded automaticly, not the script is the problem, World Wind has high CPU usage and therefore everybody has to wait so much.

canosso
Quoted post



Yes, this is another problem of your script, we must leave World Wind open and it use lot of CPU (or maybe I don't have understand your last sentence).
About the GDal, I have look at this but I don't know which parameters use to get correcte Tiff Files, actually, the world stay all flat with my converted files.

[canosso]

Hello chevket,
Originally posted by chevket@Feb 15 2005, 03:56 AM
Yes, this is another problem of your script, we must leave World Wind open and it use lot of CPU (or maybe I don't have understand your last sentence).
About the GDal, I have look at this but I don't know which parameters use to get correcte Tiff Files, actually, the world stay all flat with my converted files.
Quoted post


Did you download the FWTools: Open Source GIS Binary Kit for Windows and Linux, http://fwtools.maptools.org/, there is gdalinfo included, and the information for a terrain tile is:
Driver: GTiff/GeoTIFF
Size is 41, 41
Coordinate System is `'
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 41.0)
Upper Right ( 41.0, 0.0)
Lower Right ( 41.0, 41.0)
Center ( 20.5, 20.5)
Band 1 Block=41x41 Type=UInt16, ColorInterp=Gray

Maybe this helps.

canosso

[chevket]

Originally posted by canosso@Feb 15 2005, 02:28 PM
Hello chevket,
Did you download theÂÂ* FWTools: Open Source GIS Binary Kit for Windows and Linux, http://fwtools.maptools.org/, there is gdalinfo included, andÂÂ* the information for a terrain tile is:
Driver: GTiff/GeoTIFF
Size is 41, 41
Coordinate System is `'
Corner Coordinates:
Upper LeftÂÂ* (ÂÂ* ÂÂ* 0.0,ÂÂ* ÂÂ* 0.0)
Lower LeftÂÂ* (ÂÂ* ÂÂ* 0.0,ÂÂ*ÂÂ* 41.0)
Upper Right (ÂÂ*ÂÂ* 41.0,ÂÂ* ÂÂ* 0.0)
Lower Right (ÂÂ*ÂÂ* 41.0,ÂÂ*ÂÂ* 41.0)
CenterÂÂ* ÂÂ* ÂÂ* (ÂÂ*ÂÂ* 20.5,ÂÂ*ÂÂ* 20.5)
Band 1 Block=41x41 Type=UInt16, ColorInterp=Gray

Maybe this helps.

canosso
Quoted post



Thanks for your help,

I will try this tonight. Maybe the Type=UInt16 is the solution ?

[chevket]

It shouldn't be this, I have try rquickly to download file with the output format UInt16 and without, and in the 2 case I get the same files, which contain the same information that you provide !

So there should have something else in the software which say that there is some files to use for height or something like this...