Multiple requests for the same tile ???

[nlneilson]

I started debugging to find out why multiple request for the same tile was being made. See the attached .jpg

I ran across this in PluginSDK\Renderable\QuadTileSet.cs #1279

		public virtual GeoSpatialDownloadRequest GetClosestDownloadRequest()
		{
			GeoSpatialDownloadRequest closestRequest = null;
			float largestArea = float.MinValue;
                       //FIX -> Preliminar solution. request the same tile several times

Apparently the same tiles will be requested indefinitely.
Is this going to be corrected or has the dev that committed this code long gone?

[James_In_Utah]

It seems like the place to fix this is in WebDownload.cs in the:
catch (WebException e)
function at about line 670. The webDL argument has the path the path the was attempting to be saved. I think all you have to do is copy a blank file to that location and that should fix it. I'll try that next week at work. Right now, all the catch() does is log that there was an exception. There is also a webDL.Cancel() call commented out in that catch(). We might want to try putting that back in.
James

[James_In_Utah]

I just tried putting the webDL.Cancel() back in, and it does stop the repeated downloads. I'm not sure what other bad thing will happen because of that. Someone probably took it out for a good reason. I can't really do a lot of testing here at home. Give it a try and let me know how it work.
James

[nlneilson]

I will try that, thanks for digging that out.

edit: I un-commented WebDownload\WebDownload.cs #679, webDL.Cancel();
It still has the multiple requests.

[bull]

Good work guys keep it up, maybe Rismo took it out and planned to replace it somehow, he seems to have vanished again so any help fixing up the download code would be greatly appreciated.

[James_In_Utah]

Well, what I was hopping to do is to put a test case in that checks for error code 404, and if it is a 404, then call the webDL.Cancel(); But since that apparently isn't working for you to stop the repeated requests, I suggest we try to create an empy .jpg in the right place by the right name and see if that works. I'll look at it some more this week. We shoud be able to get this workiing...
James

[James_In_Utah]

I took a shot at it and can't figure it out. In the catch() I copied the .tmp or .txt to .jpg and tried calling the webDL.AsyncFinishDownload() and webDL.Cancel() and every other combination I could think of. WW still keeps trying to download the same file. I'm not really very up on multi threaded debugging so I'm not really sure what the problem is. Maybe someone else could take a look at this?
Thanks,
James

[James_In_Utah]

Neil,
If you are running Apache for your web server instead of IIS, take a look at the instructions Donnie put together in this post:
http://forum.worldwindcentral.com/sh...hp+tile+server
for running a tile server. You should be able to add some code at the end when he is testing for whether the file exists to return an emptyp file instead of just dieing.
James

[nlneilson]

Here is where something could be done for the multiple requests problem.
WorldWind.Renderable.QuadTileSet.cs #1202

	public virtual void ServiceDownloadQueue()
	{
            if (m_downloadRequests.Count > 0)
            {
                Console.WriteLine("    QTS #1206 " + GetClosestDownloadRequest());
                Log.Write(Log.Levels.Debug, "QTS", "ServiceDownloadQueue: " + m_downloadRequests.Count + " requests waiting");
            }
			lock (m_downloadRequests.SyncRoot)
			{
				for (int i = 0; i < World.Settings.MaxSimultaneousDownloads; i++)
				{
					if (m_activeDownloads[i] == null)
						continue;

Where I have added one line to write to the console a few blocks of code could be added for an Array of new requests. Each time a new request is made it would iterate through the Array to see if it has been requested before.

Don't confuse this with the download queue, that has been hard coded to 2 and the multiple requests need to be blocked before it gets to that queue.

One way to manage the array would be to limit the size to 100, 1000 or whatever.
A new request would iterate through the Array, if it has been requested before it would not be added. If it has not been requested before it is put in [0] and the rest shifted down and the last, if the array is full, is dropped.

In Python I would have no problem doing that. Someone proficient in C# needs to do this or find another solution.

[fevans]

Have a couple general comments, observations, but keep in mind these comments are coming from someone who is perfectly happy with 1.3.5 features, but needs a .NET2.0 release.

==========================
CalcRelativeScreenArea
==========================
The DownloadQueue uses "RenderBoundingBox.CalcRelativeScreenAre a", which relies on camera.Project(...). But the camera class was radically changed in 1.4 to stop the "jitters".

I traced through CalcRelativeScreenArea. I'm seeing positive values for bounding-boxes which are not even in the camera view frustum. Although that might not be a bad thing since, as long as the relative scores are still meaningful to QuadTileSet.

Also, the entire webrequest class hierarchy (DownloadRequest, GeoSpatialDownloadRequest, etc), the CalculateScore method isn't implemented (always returns 0). The exception is "ImageTileRequest", but isn't used anywhere.


==========================
QuadTileSet and MaxSimultaneousDownloads
==========================
for (int i = 0; i < World.Settings.MaxSimultaneousDownloads; i++)
{
if (m_activeDownloads[i] != null)
{
m_activeDownloads[i].Dispose();
m_activeDownloads[i] = null;
}
}

Why use "i < World.Settings.MaxSimultaneousDownloads" ?

Why not just use m_activeDownloads.length? If m_activeDownloads is initialized to a length of 20 (which it is in svn), and "MaxSimultaneousDownloads" is set to "2", then 18 of the elements in m_activeDownloads never get addressed. There are multiple occurrences of this in QuadTileSet.

==========================
servicepoint.connectionlimit
==========================
http://msdn.microsoft.com/en-us/libr...it(VS.80).aspx

I've tangled with download management on other projects. Even when we increased the connection limit on the server, we also had to set the servicepoint.connectionlimit in the webrequest on the client. I'm not seeing this anywhere in the WW code.

I'm not sure exactly what "Expect100Continue" does, but I've found that setting it to the "right" value can make things work.

request.ServicePoint.Expect100Continue = ???;
request.ServicePoint.ConnectionLimit = n;

==========================
Server Side config/restrictions
==========================
3. A lot of download queuing is built into IIS, and probably other webservers as well.
<system.web>
<processModel requestQueueLimit="5000"
restartQueueLimit="10"
webGarden="false"
responseDeadlockInterval="00:03:00"
maxWorkerThreads="100"
maxIoThreads="100" />
</system.web>

Clearly client-side queue management is something completely different, but things like "World.Settings.MaxSimultaneousDownloads " are also at the mercy of the server.