Tag Archives: NCDC

Scraping GISS, CDIAC, NCDC / NCEI, and Me

A collection of information about site scraping and some ideas on sizes and commands for Climate Data related sites. Continue reading

Advertisements
Posted in AGW Science and Background, Earth Sciences, Tech Bits | Tagged , , , , | 20 Comments

My, What Big Datasets You Have, Grandma NASA

Golly, what big Files you have, NOAA… and how many of them… Seems there is a “SuperGHCNd” set of data showed up on the ftp download server. About 8+ TB worth… Continue reading

Posted in NCDC - GHCN Issues, Tech Bits | Tagged , , , , , | 32 Comments

CDIAC, Compression, Squashfs, And Oddities

Some “Lessons Learned” in making a compressed easy to search archive via squashfs file systems and temperature data file trees. Continue reading

Posted in AGW and GIStemp Issues, NCDC - GHCN Issues, Tech Bits | Tagged , , , , , , | 6 Comments

NOAA data scrape completes. At last.

A major part of the NOAA data scrape completes. Some sizes and what to expect if you do it. Continue reading

Posted in NCDC - GHCN Issues, Tech Bits | Tagged , , , , , | 11 Comments