PDA

View Full Version : BUG: Mosaic Tiles with void tiles and solution


rismolath
03-17-2011, 01:36 PM
BUG:
continuing with the post:

then....I think that the problem in my old post http://forum.worldwindcentral.com/sh...ad.php?t=24264 (http://forum.worldwindcentral.com/showthread.php?t=24264) is these. I have these level distributions

----------0
--------1111
-------222222
------33333333
-----444444444
----55555555555
---6666666666666
-----7----77---7
-----8----88---8
-----9----99---9

Only a subset of level 7 exist, then server are looking for other tiles up or down. and spend very much time... and perhaps a hang problem. What do you think?

...



The problem is that when the algorithm recursively looks tiles, find a level that has a proper resolution to the request made ​​by the client and when the level is found, it looks for the tile on the disk, if not found, then it down to the next level and so on even to go down to the last level.

In the cases studied, the request of a large area causes that currentTiles (list of tiles needed to resolve the request), is composed for up to 700 tiles base level.

Later, with each tile is made ​​a mosaic in target raster, creating a loop of 700 tiles!, plus some of those tiles are not in the base level, then the algorithm goes to a higher level for tile (with an affine transform) and clip the sector to the base level. this is made with each tile of a total of 700.

This involve the server with a time of 15 seconds to solve a single request.
To solve the problem only thing to do:

gov.nasa.worldwind.servers.wms.generator s. WorldWindTiledLayer
tileMeetsRenderCriteria(ResourceTile tile, int reqWidth, int reqHeight, Sector reqSector)".

if (tile.getLevel().getTexelSize() > reqTexelSize || tile.getLevel().isEmpty())
{
return false;
}
else
return this.dataFileStore.containsFile(tile.get Path());

change to
if (tile.getLevel().getTexelSize() > reqTexelSize || tile.getLevel().isEmpty())
{
return false;
}
else
return true;

This change ensures that the algorithm returns the list of tiles ideal would resolve the request, then another algorithm (which performs the mosaic) takes care of looking if a tile exists or not and to find its equivalent in the pyramid up .

This change has improved the speed of my server from 15 seconds to 0.8 seconds. to requests for large areas and irregular data pyramids where some tiles do not exist, can mix in a layer different images of different resolutions and levels:

Use Case:

We start ICUBED, and as we have better resolution images of a given area of the world, mix with these ICUBED images obtaining:

----------0
--------1111
-------222222
------33333333
-----444444444
----55555555555
---6666666666666
-----7----77---7
-----8----88---8
-----9----99---9

This change improves server response to requests involving tiles that do not exist in the repository, both for requests of 512 x 512 on WWJ and for larger requests made ​​by other clients ESRI, ERDAS, etc..

Regards
Raul

nlneilson
03-17-2011, 03:37 PM
This change has improved the speed of my server from 15 seconds to 0.8 seconds.

to requests for large areas and irregular data pyramids where some tiles do not exist,

I have read your posts on WMS servers. The speed increase caught my interest.

I have clipped, merged and tiled the FAA Charts. After tinkering with a WMS and after testing have "served" them for several years as an "external cache" (no WMS) without problems and very fast. To make them more accessible to a wider base have been considering making them available as WMS requests also. Your indicated speed increase may make this a viable option.

The major slow down I perceived with a WMS was when the images were not pre-tiled and that had to be done on the server when requested.

If I decide to go this route and see much of a slow down may have some questions.

You have made good suggestions.

Neil

rismolath
03-17-2011, 04:22 PM
hi neil

I have created a global mosaic, based on iCubed, with several areas to another resolution
Dstile using my version, I generated the mosaic levels ICUBED from 5 to 9, in order to resolve queries (180.90, -180.90) and other very large coverage, with few tiles

In this mosaic, generated by zones, with better resolution images, right now I have only published ICUBED layer and areas of Spain to 50 cm (PNOA), areas around the world 0.7 (QuickBird) and spot areas (2.5m ) Mixed with iCubed, so that, in more detail the structure of tiles and levels is:

----------0000
--------11111111
-------2222222222
------333333333333
-----44444444444444
----5555555555555555
---666666666666666666
--77777777777777777777
-8888888888888888888888
999999999999999999999999
-----AAA----AAA---A---A----
-----BBB----.BBB---B---B----
-----CCC----CCC-------------
-------------DDD-------------

With this configuration, the WMS server works fine but without the change I describe, a request for a high resolution and wide coverage level "D", and if this coverage is out of the existing tiles, generates a list of tiles to solve it very big, and the algorithm takes 15 seconds with the change described, justified in the manner explained, is reduced by 10.

If you need any help on these issues do not hesitate to tell me.

Greetings
Raul

nlneilson
03-17-2011, 05:13 PM
Hi Raul

I just started looking at more of the threads in the Server Discussion forum.
I don't recall what I did before as some time has past and the only tinkering was done with a local server before having a Web site.

What is needed for the .xml for a client and how that can be retrieved from the server with getCapabilities and the changes required in the code for the layers has been covered in different threads.
It will just take time finding them again.

If you know of a link that ties all that in with the WWJ Server code that would be a time saver on my searching.

As mentioned my site as far as the tiles is basically set up as an external cache.
http://www.nlneilson.com/serv/FAA/48States/
http://www.nlneilson.com/download/FAA%20Sectionals/Sec_48.java

Neil

rismolath
03-17-2011, 06:17 PM
Ok, try to explain here, follow these steps

1 .- download the latest version of the server:
"Http://builds.worldwind.arc.nasa.gov/download_server.asp"
2 .- unzip the downloaded file to a folder
3 .- inside the folder you will find:
<doc> -> javadoc Documentation
<lib> -> classes needed to operate the server properly, including classes in a version x worldwind.jar
<src>server source code
<WEB-INF> configuration files, this is most important to mount the server, without searching for more problems
README.TXT -> information that can be very helpful.
startWMS.bat -> windows server starter (console mode).
startWMS.sh -> linux server starter

4 .- Now you have to create a folder where you will want to publish data, typically WorldWindData I call it, create it wherever you want

5 .- then gets inside that folder to publish data

For example and start with a simple:

<PATH> \ WorldWindData \ ICUBED

inside the folder, leave the whole structure of typical WW tiles, level, latitude folders, tiles.jpg. So you will have:

<PATH> \ WorldWindData \ ICUBED \ 0
<PATH> \ WorldWindData \ ICUBED \ 1
<PATH> \ WorldWindData \ ICUBED \ 2
etc ...
<PATH> \ WorldWindData \ ICUBED \ level-n

6 .- within this folder, ie:
<PATH> \ WorldWindData \ ICUBED
you leave a xml configuration file regardless of the name. with the same content as the file that generates the client WWJ for that layer, ie the one that makes windows:
C: \ Documents and Settings \ All Users \ Application Data \ WorldWindData \ Earth \ NASA Landsat WMS I3

edit it to change web addresses that you feel should be directed to your server, and the parameters that apply to your data.

7 .- Now let's configure the server end

<WEB-CONF> \ wms.datafilestore.xml
reads this file, try to understand (is easy), and add your directory WorldWindData where are all your data.
<WEB-CONF> \ web.xml
configure the service ports

8 .- else you can leave the default for your first test, now runs the startwms.bat and read what I say in the console, if everything looks good, you will have your server with your first data.

9 .- there are things that can be set from there.
GDALPATH for other layers (not tiles) -> a good idea is to use FWTools bin folder and GDAL_DATA environment variable

The rest of the settings, layers, elevations SRTM,DTED, etc. if you want to discuss them another day.

there are many parameters that are configured by default for this type of configuration, and you should not touch them, so if you do not play anything other than what I mentioned should work correctly.

The server is preconfigured to run self-discovery tiled layers. The server searches the folders defined in datafilestore seeking information about xml that publish data, if not set the xml file in the place where I said, the server will search all subdirectories of WORLDWINDDATA in search of xml and maybe hang.

I hope not to have been wrong in anything, (I'm doing it from memory), if so tell me

Greetings
Raul

nlneilson
03-17-2011, 06:34 PM
Thanks Raul, that will be a great help.

I was looking at this link but it may be a bit outdated.
http://forum.worldwindcentral.com/showthread.php?t=24183

You made a very good explanation.
Will let you know how it works.

Neil

nlneilson
03-19-2011, 12:58 AM
Hi Raul

With your directions I am getting somewhere.
INFO: Application NASA WorldWind WMS Imagery Server registered successfully to port 8,000 and virtual directory /wms

The only thing I had to change was in wms,config.xml was:
wms.NLNwwDataFileStore.xml
and then paste in the .xml files I had into the proper places.
When I get some results/errors will make changes as necessary.

Neil