Daily Challenge Day 24

Well so far, whilst messing with maps, not had a problem finding stuff to write for the daily challenge. So around 2:20 am this morning I generated a revised map in ProgeCAD. Not so easy. I went back into google earth to add Port Wakefield as a reference and some additional towns between Adelaide and Jamestown. I added these to my existing collection of towns, but noticed that the names weren’t hyperlinked as with the original.

The original towns I got by creating a minimum kml file with addresses, then loading into google earth, once loaded google earth inserted pins with surrounding rectangles. When saved back to a kml file, the file then contained coordinates for each location: I believe this is called geocoding. For the additional towns I used the map pin from the toolbar in google earth.

Anycase i exported the kml file and tried to open in MS Excel. It wouldn’t open so I had a guess that it was due to the difference between the points. So I looked at the kml file with a text editor (ultraedit), and noticed that the place markers did have different record structures. The record for the batch geocoded points contained a “MultiGeometry” element with the point coordinate inside that. The records for the manually inserted pins contained a “lookat” element, but the point element was on its own at a higher level. Thus I reasoned that the problem was different record structures and therefore Excel unable to form a single table. I therefore went back into google earth and separated the points into two groups. Each of these kml/xml files was able to be opened in Excel, so I was able to then copy/paste the data into another workbook to create a csv file for import to QGIS.

When I first installed QGIS I tried importing the kml file directly but didn’t have much look, once again I assume it is because the kml file is too complex with too many variable record structures. So I found that csv files are easier to work with, once remember how to import them: I had to do a google search to find out how. Once imported it’s easy to export, if remember how, once again I had to go lookup some guidance. It is not very clear that the projection is for the export conversion: given that else where have to pick the starting projection and then the projection want to transform to. Anycase once figured I exported the data to AMG coordinates and imported back into Excel.

Once back in Excel I compared against my existing list. No change there, so conversion of additional points compatible. Identified 17 towns which had not been geocoded. also some which had been geocoded but missing from the original list so added them along with the extra towns. Built a new list ready for generating map in ProgeCAD.

Generated the map in ProgeCAD without problem, so decided to update the place marker block so that attribute text on another layer. Ran my Excel/vba macro again and generated my map. Saved the drawing and swapped some filenames around, as the drawing with the towns is externally referenced (xref) into my main map drawing. Opened my main map drawing, the extra towns seemed to be relatively in the right place. Went searching for the previously missing towns, used find on the text, but didn’t find anything. Then realised couldn’t access the attributes in the xref, so went back to the town drawing and searched. Then remembered why the towns weren’t previously included, they were obviously wrong: Port Vincent was located in Yorketown.

This was noticed in the first instance because part of the reason for creating the map in the first place was as a starting point to investigate the logistics between Yorketown in the South with a population of around 5000 people, and Kadina in the north with a population of around 5000 people. Maitland some where around the middle of the two, with a population of 1000 people, and most other towns with smaller populations. The Yorke Peninsula council area has population of around 12,000 as does the Copper Coast council area, but the Copper Coast council area is a fraction of that of the Yorke Peninsula area. Kadina is in the copper coast council area, Yorketown and Maitland are in the Yorke Peninsula council area. As a rough observation Maitland is a lot closer to Kadina than Yorketown. Maitland has a hospital and a nursing home, next to one another, yet people in the nursing home are sent to Wallaroo hospital and elsewhere. So something doesn’t seem right with the use of the available resources, and with government cut backs its seems it’s likely to get worst than better.

so anycase, the location one of the towns was noticeably incorrect, so whilst the export/import was fresh in my mind I thought I fix the location. So I went back into google earth and attempted to move the pin: not possible. The manually entered pins could be moved but not the batch geocoded pins. Something definitely different about them: but at 2:20am it seemed like time for sleep and something to look at later.

Roll around 11:05 am. I open google earth and check what the problem is: it mentioned something about multiple elements and only being able to edit the first. ahh! I saw something like that earlier in the kml file. Go look at the kml reference, for “MultiGeometry”, “LookAt” and “LinearRing”. Whilst “LookAt” contains a latitude and longitude, these are not needed because the place marker also contains a point with coordinates. As for multigeometry, I don’t know what use the linearring is: I assume that as some places are localities the ring approximately defines a boundary or at least could do so. So rather than add “LookAt” to the geocoded data to make them look like the manual pin data, I could create simple place marker records with just points, names and addresses. So I copied the kml file, deleted all records except two, and imported as a test into google earth. Sure enough the pins could now be relocated. I exported the file and it remained simple no extra complications added.

Then the fun started, or rather didn’t. Manually editing all the records in the kml file to a simplified form using a text editor, seemed like a waste. Since Excel could read the file it seemed better that could open in Excel and delete the necessary records and then save to another file. Deleting was easy, saving back to kml/xml was not.

Creating the xml map file was a major hindrance. I’m relatively show last time I looked at it, i found a website with a simple line of vba code to enter in the immediate window in VBE, to list some data which could be cut and pasted or something. Couldn’t find that. I attempted to write a schema file, but it would only accept a simple example, as I made it more complex it ejected: something invalid about the use of the tags. Not sure what the problem was, as the file checked as valid xml, so issue was something invalid about the record structure attempting to define. The other problem was that with the xsd file it could read, I was unable to map the xml structure to the data, I could only map to the header.

An internet search merely indicated how to use an xml file as a template, and the need to include at least two records for repeated elements. Which seemed fine if using an xml file, but what about using a xsd file? The gist of most of the websites seemed to be use the xml file as its more reliable. So having searched for xml editors and other tools to create xsd files from xml data, and scrapped that idea as way too expensive for occasional usage. I gave up on trying to produce an xsd file, and simply went with creating a simple xml template file: which I guess accounts why I don’t have a valid xsd file for the previous imports/exports.

So by teatime around 18:00, I finally had a kml  file to import back into google earth, with pins I could edit. Besides Port vincent being way out of position, most other towns at a guess were around 10km to 25km away from proper location, based on my relative observation of having spent time looking at my token map with 25 km grid. Some pins were in the middle of the ocean of the coast of Africa: I assume these were the pins without coordinates. Some towns and localities I could not find, so the pins may or may not be in correct locations.

As for the correct location, what is it? Most of the references on google earth, the red dots, are located on highways, so not exactly in the towns but on roads passing them by. Some localities are in the middle of nowhere, so not really towns but just a  generally named area: possibly large farm or something. So some of these places cannot actually be visited, there are no roads to them and the location seems like some arbitrary point. Even points for towns seem  arbitrary, some are at post offices, a reasonable tradition, others seem at road intersections at random points in and around the towns.

Therefore ultimately in defining the list of towns for a looped journey taking actual roads into consideration, the places need to be somewhere specific in the towns. For example in Kadina the town square. As for Maitland, no idea, it has a main highway charging through the town centre: people entering the town with their caravans also seem to have no idea. Maitland seems to be some place to pass through on the way to somewhere else, after all its not on the coast: no beach or nature walk. What else would you do driving along the Spencer Highway: state route B89: keep going until leave the highway?

So anyway, I’ve still got to filter out the towns/localities I cannot find verifiable coordinates for on the underlying google earth maps. Also strange that geocoding didn’t put the pins directly on the existing red dot markers, since they have to come from somewhere. So seems need another reference to verify locations mapped via google earth just to get that extra bit of confidence that everything is in reasonable location: if not exact.