Large scenery - plane dive

[NickB]

I've merged 3 sceneries into a large one (27x26) and ran the command line with LandscapeEditor.exe -hash Ridge-NSW-2 and it creates Ridge-NSW-2.tha and I run it again and it creates Ridge-NSW-2.fha. I can set-up a task and fly and everything looks good, but after a few minutes, the glider dives at the ground. I did create the hashes, so is there some other limitation ?

[NickB]

After some analysis, it becomes obvious that when you exceed 25x25 tiles that duplicate filenames occur when leading 0's are dropped. File 101,00 (column, row) and file 10,100 have the same name 10100. The 'hash' however uses 6 digits, not just 4 or 5, and must use the filename in its hash calculation as well, because the hash is different for 010100 and 101000 even though they point to the same file 10100.

With 27x26 tiles, 4 files have duplicate names and thus are overwritten during the re-indexing of files. This means there are overwritten areas in the landscape imagery, terrain and forest, and because the hash table uses 6 digits for names, and includes its filename(?), the hashes are present for both, but they are different.

Just in case Condor perhaps tries looking for files with the leading 0 first and then without the leading zero second, I tried to add the missing files with leading 0's in their name, i.e. for 101,00 use 101000, or for 10,100 use 010100. Didn't work; the hashes were the same. It would be nice if that file presence check could be added to Condor-2 to eliminate the issue. (if file 010100 exists, then use it, else look for file 10100).

I also tried to remove the overlapped files to make the hash 0 to see if the hash test fails or not and the plane dives. The hashes were indeed 0 and thus matched, but it still didn't work, for some other reason I guess.

For Condor-3, perhaps the leading zero check-first could be added, or use Hexadecimal to keep 4 characters and range 256x256 instead of 100x100, or switch to 6 digits.

So the solution for now is to limit the rows to 25, or the columns to 25.

Another issue is at the seam where two sceneries are merged/spliced; there is a small elevation issue showing as a white area with an elevation change. I guess that is why the TR3 files are 193x193 with an overlap instead of 192x192. Either the initial TR3 files must have the proper overlap when initially created, or the overlap needs to be corrected when sceneries are merged.

[Xavier]

Hi NickB,

Your guess is not true: AA2 is 40 x 25 tiles and CA is 32.5 x 47.5 tiles.

have a look to the notations of patches in their respectives "Textures folders" or in LE and apply your findings to your scenery.

Glider diving can also be find when Condor 2, during simulation, try to load missing heightmaps patches (hxxyy.tr3) or when it try to load corrupted heightmaps patches.

Good continuation.

[NickB]

I had checked AA2 and CascadeRange, but not CA.
- AA2 is 40 x 25, so it's OK because the row count 25 is less or equal to 25
- CascadeRange is 25 x 45, so it's OK because the column count 25 is less or equal to 25
- CA is 32.5 x 47.5, so it exceeds 25 in both column and row
- the file names have the same format as I have used
- it should have 4x32.5 x 4x47.5 = 24700 files, but it has only 24432 with 268 missing due to duplication
- this is similar to the 4 files that are missing on mine due to duplication
- because of the missing files, the scenery has bad imagery, forest, terrain, and you can see columns of bad tiles
- this is similar to my scenery due to the missing files
- but if it works, what is different ?

[NickB]

Well, there is no difference. It turns out that missing TR3 files were the issue. Thanks Xavier for pointing out this possibility.

So, there is still a defect because of the duplicate overwritten files. In the case of CA (California-2), the defects (238 duplicates) are over the Pacific Ocean, so it is not a big deal. In my case, I have 4 duplicates, 2 on the scenery edges which are not flyable anyway, and for the two others I'll split the defect to opposite ends of the scenery to have only one defect in each area.

It would still be nice if Condor-2 could check for the presence of a 6 digit filename first, so that this defect could be eliminated.

I still have to figure out how to deal with the small elevation defects on the seams of the merged landscapes.

It also turns out that the LE (LandscapeEditor) -hash only needs to be run once. Although the command prompt returns quickly, the LE is actually still working away for a couple of minutes. You just have to be patient and wait for the files to show up in the folder.

Ridge-NSW-2-3-merged sceneries. jpg

[Xavier]

That' fine NickB,

Now just make or find a tr3 file with a contant altitude of 0 m or something else.

The fastest way is to duplicate one of your tr3 file rename it for an missing file, and in LE make it flat. Then you have only to duplicate and rename your new tr3 files at the the border of your scenery with 2 patches in all directions. I guess that nobody want to fly in the dark side!

It is the way I solve the issue when I encounter this kind of difficulties in one of my sceneries. You can also add textures of your own.

Good luck.

[dgtfer]

- CA is 32.5 x 47.5, so it exceeds 25 in both column and row
- the file names have the same format as I have used
- it should have 4x32.5 x 4x47.5 = 24700 files, but it has only 24432 with 268 missing due to duplication
- this is similar to the 4 files that are missing on mine due to duplication
- because of the missing files, the scenery has bad imagery, forest, terrain, and you can see columns of bad tiles

- but if it works, what is different ?

For Condor-3, perhaps the leading zero check-first could be added, or use Hexadecimal to keep 4 characters and range 256x256 instead of 100x100, or switch to 6 digits.

I hope that the XXXYYY format will be implemented in C3. So C3 sceneries size could theoretically reach 5 760 km in each direction, far more than what memory size and UTM distortions would allow!...

For now, transgressing the 100 dds limit create a confusion zone. It is only possible if you have the duplicate tiles in an unreachable part of the scenery, like in CA. You must also tweak your .trn file, flattening manually the duplicate part to avoid the creation of weather events (thermals or wave) that could reach the flyable zone.

On Condor 1 we could have bigger sceneries like "Vinon-Fes" flyable everywhere with a small batch program that was swapping the various tiles according to the pilot's position, but that's not possible now, with the anti-cheat mechanisms.

So in your case, as your scenery has a confusion zone over the land, you can't solve directly the problem...

But, there is still a slightly dishonest but nonetheless efficient solution...
You could rotate artificially your scenery, for example by reprojecting it on a western adjacent UTM zone. I think your scenery is presently on UTM 18, so UTM 17 or even 16 should do the trick!
That method avoids decalibration problems, and external softwares like seeyou or XCSoar are still perfectly accurate for position.
Of course, it increases the distances errors, but that effect remains very small, the most significant effect is the "projection declination". It causes a rotation that will add (or subtract) to the local magnetic declination.
But I also used that method in Vinon-Fes, and this effect was absolutely unnoticeable in flight.

Capture.JPG

[NickB]

Thank you Xavier and dgtfer,
I had thought about making all edge tiles with very lower contrast to make it obvious it was a no go zone. Black would be more obvious.
I thought about doing a 'squeezed calibration', and I thought about rotating the whole scenery, a kludge similar to a Condor V1 scenery that was above 60deg North and moved south, but the lat/long would not work compared to real life for attached PDAs, but I hadn't thought about using an adjacent UTM zone. An interesting idea.
I read up on Xplane and how the scenery is done in 1 deg vertical and 1 deg horizontal areas and how you can basically go anywhere on the planet and just have enhanced areas developped as desired. This seems a better approach in the long term, than using UTM for selected areas, and having to do large sceneries because you can't cross boundaries.
I only have four problem tiles to deal with and they are on the edges, and far away from the terrain of interest along the ridges in the centre, so it is not a big issue.
I wanted to write an application that was able to merge select sceneries, so you could effectively fly across boundaries if you wanted, but with UTM, it's limited to the same UTM zone and same UTM grid reference. I created the 3 initial sceneries with that in mind, and it tales about 20 minutes to merge all 3 into one, with missing areas filled with default data. I just need to finalize how to fix the seams. If all scenery designers could agree on a default UTM grid (5760m squares) in any UTM Zone, any nearby group of sceneries could then be merged (and cropped) relatively quickly, on an as needed basis. No re-projection needed, just tile re-indexing.

[dgtfer]

I wanted to write an application that was able to merge select sceneries, so you could effectively fly across boundaries if you wanted, but with UTM, it's limited to the same UTM zone and same UTM grid reference. I created the 3 initial sceneries with that in mind, and it tales about 20 minutes to merge all 3 into one, with missing areas filled with default data. I just need to finalize how to fix the seams. If all scenery designers could agree on a default UTM grid (5760m squares) in any UTM Zone, any nearby group of sceneries could then be merged (and cropped) relatively quickly, on an as needed basis. No re-projection needed, just tile re-indexing.

An automated merging, even from different UTM zones, is possible if you master Global mapper scripting for example.
But while the framework merging needs only a few hours, you'll still have a lot of work to prune the object file and manually retouch the textures.

[NickB]

I was thinking of something simple, relatively fast, not requiring re-projection which would require the original imagery, not just the DDS imagery.

[Xavier]

Hi NickB
To finish your scenery and avoid the white surfaces where the tr3 altitudes mismatch between two patches, here is what you have to know. It is unofficial and undocumented. So work carefully and only with a copy of tr3 files.
A tr3 file is a binary file with no header in Little Endian format which covers a patch. That is that the first 2 bytes are the altitudes in metres in hexadecimal inverted, and so on, starting from the bottom right and upward from column to column.
As the border columns and rows are the same with adjacent patches the number of columns and rows are (5760 m/30 m) + 1 = 193.
So a tr3 file is a matrix of 193 x 193 altitude values and no more.
Here is a short explanation that explains how it works:
I open the h0228.tr3 files of one scenery in a binary viewer and click on the first value. The first 2 bytes are "6B" & "16" in little Endian that is the number166B in hexadecimal so 5739 m.

tr31.jpg

If I open this scenery in LE and zoom to the patch 0228 (PatchX =2, PatchY = 28) with Height map checked, the altitude displayed at the low right corner is 5739 m.

tr32.jpg

And if I select the 3rd byte in the binary viewer, the third and fourth values are "6F" & "16". That is 166F or 5743 m.

tr33.jpg

Etc , etc. ..
Now you have all the clues to avoid the westward white surface.
Select the 193 x 2 = 386 ending data of the hxxyy.tr3 file.
Copy this data in the clipboard.
Then open the hxx+1yy.tr3 file and replace the first 386 data by the data in the clipboard.
Good luck

[Xavier]

Consecutively the difficulty seems to find the last 386 bytes of data in the first patch and their ending place in the second patch.
It is really quite easy:
Let's have a look to the positions of first 386 bytes and the last 386 bytes in a tr3 file according to their addresses in the file and their value and position in LE to demonstrate what to do exactly.
I open the same h0228.tr3 files of same scenery in a binary viewer and change the address column to decimal value. In LE I zoom over the same scenery to the upper right corner of the same patch with Height map checked, the altitude displayed at the upper right corner is 5666 m.
In the binary viewer the address of the 386th bytes must be 000 385 because the first address start at 000 000 and the last 2 bytes value address start at 000 384. If I select this address, the answer is obviously 5666 m in the binary viewer.

tr36.jpg

So now the first column of 193 altitudes values of all tr3 files is always between addresses 000 000 to 000 385 dec (In yellow in the picture)

Now to determine the last 386 bytes of the tr3 files we proceed in the same way for the lower left corner and the upper left corner. Look at the evidence in the next 2 windows captures

tr35.jpg

tr34.jpg

So now the last column of 193 altitudes values of all tr3 files is always between addresses 074112 to 074 497 dec (in yellow in the picture).

In the Binary Viewer the size of tr3 files is also displayed: 74498 bytes that is 193 x 193 x 2.
The last address is 074497, but adresses begin at 000000 and not 000001.

Good luck

[NickB]

Thanks for the information. I know about the overlap on the top row and left column of each TR3 file. I wrote a function to fix each seam.
// copy bottom of top file to top of bottom file
// i.e. first column, to last column
for k := 0 to 193-1 do begin
FileIndex := 0 + k*193*2;
Seek(Data_File,FileIndex);
BlockRead(Data_File,P^,2);
Seek(Data_File_a,FileIndex + (193-1)*2);
BlockWrite(Data_File_a,P^,2);
end;

// copy right of left file to left of right file
// i.e. first row, to last row
FileIndex := 0 + (193-1)*193*2;
Seek(Data_File,0);
BlockRead(Data_File,P^,193*2);
Seek(Data_File_a,FileIndex);
BlockWrite(Data_File_a,P^,193*2);

There remains a problem in the duplicate/missing tiles that simply cannot be fixed.