2005 September 18 w/Tom Barclay
Meeting with Tom Barclay at EIL
next meeting
- Attendees: Tom Barclay, Mike Colee, Peter Slaughter
Jeff had the following topics he asked Peter and Mike to discuss w/ Tom (The 3 of us were not certain of how to interpret all of these so we didn't necessarily do a great job answering them.)
- Implementation of TerraServer V6.0 for storage of our data (get copy of the V6.0 spec).
- MTC: My impression is that Tom does not advocate storing everything in a DB
- Tradeoff in Tile sizes and implications for database (small tiles -> lots of objects and therefore put in DB, whereas large tiles mean fewer objects so file system can handle).
- Mechanism to use TerraServer for the input. I think Tom has set this up.
- Conversion of our h/w to support SQL Server on cluster (i.e., windows not linux).
- MTC: It is unlikely that FusterCluck would be rebuilt as a windows cluster since Davey's group paid for it and wants it as is.
- Thus, a windows based cluster would require a new slug of $$$.
- If we are instead interested in load-balancing, we'll need to invest in some (2+) disk bricks to be built up w/ the Windows OS.
- After discussing w/ Jeff, first step would be to bring some shuttles online running Terraserver pointing at the existing disk.
- The advantages of doing this are that Tom has thought out the issues and implemented, so we can spend our thinking and coding cycles on other issues. Clearly TerraServer handles the size problem, as their application is bigger than ours. And we better utilize the partnership with Microsoft.
- Disadvantages are ??
- This attachment was also part of Jeff's email.
Tom spent a few minutes discussing 'Virtual Earth' (http://virtualearth.msn.com)
Some highlights:
- contains 18TB of data
- 2-3 TB of spatial indeces that are kept in memory (indeces are NOT in SQL server)
- other file metadata is in SQL server
- uses spatial quad tree
- Direct-X supports WGS 84, so can do triangulation
- all data is in native projection
- can reproject to desired projection
- the following resolutions are stored: 16m, 8m, 4m, 2m, 1m, .5m, .25m, .125m, .01m
- different 'viewers' are available
- MTC: Runs on a "load balancing" pool of hardware, not a beowulf cluster
- Microsoft's term for beowulf clusters is "HPC" and Tom said he may be able to hook us up w/ that software tho he's never used it an it's very new.
Tom emphasized that he's not advocating using TerraServer or VirtualEarth unless it makes
sense for the needs our project. We spent some time discussing what those needs are. It's clear that
this hasn't really been defined yet and needs to be. Some points that we discussed:
- we have multiple regions of interest (ROI) that each have a different projection, possibly with 'custom' projection parameters
- do we tile all our data in one 'native' projection for all ROIs
- how could we reproject to 'custom' projections - TerraServer doesn't support this (I think)
- MTC: got the impression that Tom felt that tiling schemes should conform to the original data.
- MTC: Also that data and products should be stored/processed in original projections to minimize errors introduced thru repojection process. 3rd re-projection generally results in trash ("pixel raping").
- we have multiple product types for each ROI
- MODSCAG snow grainsize, snow map, veg map, rock map
- AVHRR snow product
- redistributed MODIS: several products
- how are the different products and coverages displayed
- Tom uses icons on tiles to achieve this
Issues to clarify or address w/ Tom:
- Get his tiling spec spreadsheet
- Look into MS HPC software if we want a windows based beowulf
Tom is willing to come down after Oct. 15 to discuss 'Snowball' issues.
Links to this Page