 |
| The
Red Mosque of Colombo. If you are not sailing nerd, this is the last
interesting thing you will see in this post. Run away now. |
Your friendly cruise director, Jen, has taken the day off. Unfortunately for you, you're left with Matt to fill the blog void. And this week, instead of a relentless recap of our boring lives (especially here and now, stuck in Galle), 99.9% of you are going to get treated to something you've probably never wanted: A tutorial on how to convert cache files in SAS Planet. If you've never heard of SAS Planet before, this is probably your cue to shake your head, close the browser, and try your luck again in a week or two. If however, for the remaining 2 or 3 of you, you DO know about SAS Planet, then stick around.
UPDATED Dec 1, 2020- Check the bottom for another option in addition to what's detailed below.
Why you
might want to convert your cache structure:
SAS Planet ("SAS") is a wonderful
tool for offline map and imagery viewing. It allows for virtually unlimited
storage of all sorts of map types (Google Satellite, Navionics, Bing, ARC.GIS,
and many others). It also allows for
integration with a GPS feed to show current position, speed and heading
overlaid on any of the stored maps.
However, as
the number of your stored maps grows, the storage container for the maps, or
"cache", becomes unwieldy and difficult to move, copy, or share with
others. This is because the original
cache structure (assuming you are using the standard "SAS.Planet"
cache type) was set up on a series of nested folders to organize and contain
the literal millions of small images that are stitched together to create the
maps shown on the computer. Each image
file is only 1-2 Kb, but a good sized SAS cache might contain 10 million of
these small files. The problem arises
when you go to backup or copy your cache. Computers are becoming faster and faster with passing data, but the
problem here lies in the overhead associated with the Windows (or any other)
operating system.
When
copying a file, you, the user, only
worry about dropping and dragging, or hitting Ctrl-C then Ctrl-V. However
behind the scenes the operating system is having this sort of conversation:
OS: Hey, Disk, User wants to copy
file x to location y.
Disk: OK.
OS: Does that location exist?
Disk: Nope
OS: OK, create it.
Disk: OK, created.
OS: Are you ready to receive the
data?
Disk: Yes
OS: Here's the data [this is where
the 1.8 KB gets sent along]
Disk: OK, got it.
OS: Did you get all of it?
Disk: Hold on, let me check... Yep,
it's here.
OS: OK, that's it. Goodbye.
When
copying a large file, that "conversation" overhead is almost unnoticeable. However, when copying millions of small
files, that overhead is where the computer spends most of its time relative to
the small image file that it passes along. Consequently, copying or backing up
a 10 GB cache might take hours, whereas
a 10 GB movie file might only take a minute or two.
Luckily,
more recent versions of SAS have provided support for a different cache format
known as "SQLite3". Instead of storing millions of individual image
files, the cache has a limited nested directory structure which ends in one or
more small database files. Each database
can contain thousands of images.
Let's use my Google Satellite cache as an example. When converted to a SQLite3 format,
the cache went from 3,406,505 Files and 114,444 Folders to 1,681 Files and 992
Folders. That is a whopping 99.96% reduction in the number of files! This allows the operating system to copy and
back up the cache files much more efficiently.
How to
convert your cache:
So now that
you understand the need to convert the cache, let's get in to how to actually
get it done. Within SAS, there is Cache Conversion utility. However, it's rather manual and takes a bit
of setup to get it to work properly.
That said, it's not terribly complicated and the payoffs are large, so it's
worth doing.
The first
thing to understand is that the SAS Cache Conversion utility only allows you to
convert one map source at a time. So,
depending on the number of map types in your cache, you may need to repeat this
process several times. Also, to prevent
confusion, it's best not to have SAS attempt to download more new files while
you're are doing this process, so for now set your data source to "Cache
Only".
Step 1- Navigate
to the main SAS directory. This is
easily found by opening Windows File Explorer and navigating to your SAS Planet
folder. In this main directory you
should see a sub-directory called "cache". This is where your existing cache files
reside. Create a new
directory called "cache_sqlite". At this
point you should now have a "cache" sub-directory and a "
cache_sqlite" sub-directory. You
only need to create this "cache_sqlite" directory once. This is the new directory where all of our
new SQLite3 map specific caches will reside.
 |
Proper set up of the directory structures is the key to a successful conversion
|
Step 2- Take a
look in your "cache" folder
Within "cache" you should see many subfolders with names like
"sat", "both", "navionics", "vesat",
etc. Each one of these sub folders represents a different map type that you
have downloaded at some point. Most
names are pretty self explanatory, but a few are more cryptic. Here are a few you may not immediately
recognize:
- "sat"
--> Google satellite imagery
- "Both"
--> Google Hybrid images (this is the overlay that shows town names,
streets, businesses, etc.)
- "vesat"
--> Bing satellite imagery
Determine
which map type you want to convert. Let's
use the Google satellite imagery as an example. Drill down into the sub-directories under that map type ("sat")
until you reach the final level that contains the image files. Make a note the type of image file that the
map type uses (possible to be .jpg, .png, .bmp or even others). Google satellite imagery uses .jpg. Come back up to main SAS directory.
Step 3 - We now need
to make a new sub-folder in our "cache_sqlite" directory that is the EXACT same name as the sub-folder
in the "cache" directory that we want to convert. So if we are going
to convert Google Satellite imagery, we need to create a "sat" sub-folder
within our "cache_sqlite" folder.
The sub-folder names must be identical or the SAS program won't
recognize the new location.
This
completes the "set up" required, and we are now ready to start the
SAS Cache Conversion utility.
 |
Conversion utility screen
|
Step 4 - Within
the SAS program, navigate to Operations -> Cache Manager
This will
bring up a bring up a window where we need to enter some info for the
conversion to work properly.
·
In
"Source Cache", leave the first drop-down set to "Folder".
·
In
"Path", click on the ellipses button to the right of the blank field
and navigate to and select the source folder, namely your "sat"
subfolder within the main cache folder.
·
Set
the "Format" drop down to "Sas.Planet"
·
Set
the "Extension" dropdown to ".jpg" (this is the file type
you discovered when you drilled down into the cache in Step 2 of the set up
process)
·
Ignore
all the other fields in the top half, or "Source Cache" area.
·
In
the "Dest Cache"", leave the first drop-down set to
"Folder".
·
In
"Path", click on the ellipses button to the right of the blank field
and navigate to and select the destination folder, namely your NEW "sat"
subfolder within the newly created "cache_sqlite" folder.
·
Set
the "Format" drop down to "SQLite3"
·
Ignore
all the other fields in the bottom half, or "Dest Cache" area.
You are now
ready to run the conversion. Click the "Start" button. A progress window will pop up. It will take a bit of time for the system to determine what it needs to process, but
after a few minutes (possibly longer if your cache is really big), the progress window
will start showing how many files it has processed. Just for planning purposes, a large cache
of roughly 6 million files (58 Gb) took almost 13 hours to process. However
this was on a machine limited to USB2 and the cache was on an external disk. I
would assume that the ability to use USB3 would have sped things up
considerably.
 |
Setting up SAS to use the new SQLite cache type.
|
Step 5 - Once the
process is complete, if you want to check that everything went smoothly, you
need to tell SAS to start using the new cache. This can be done as follows:
·
Navigate
within the SAS program to Settings -> Options
·
Select
the "Cache" tab
·
Make
sure the "SQLite3 cache folder" is set to "cache_sqlite"
·
Change
the "Default cache type" dropdown to "SQLite3"
·
Click
"Apply"
You should
now be able to see your newly created cache. At this point you should ONLY be able to see
the one mapset you just converted (make sure the selected basemap is properly
set). All the other map types will be
grey, as that data has not been converted yet.
Step 6 - To
finish the conversion for other map types, simply repeat steps 2-4 as needed.
Phew! Did you really read all that anyway? If so, I gotta suggest you find yourself a hobby.
UPDATE Dec 1, 2020:
Another option for this process is to use the program called "Sat2Chart" developed by Paul Higgins. The program is primarily used to create .kap and mbtile charts from imagery sources like Google earth and Sas Planet. However, Paul has also recently included a utility that automates the cache conversion process detailed in my post. The utility is called "Convert_Cache" and is found within the "Sat2Chart" program. You can download the program here: http://gdayii.ca/Downloads/
To run Sat2Chart, you will also need to download and install OoRexx 5.0.0, which is also available from the download location given above.
When the Convert_Cache window opens, there is a "Tutorial" button you can click that walks you through the operation and setup of the utility.