Home > GHCN > GHCN v2.temperature.inv with GPW3 population density

GHCN v2.temperature.inv with GPW3 population density

2010 March 3

Gridded population density data (pop/km^2) is available at http://sedac.ciesin.columbia.edu/gpw

Center for International Earth Science Information Network (CIESIN), Columbia University; and Centro Internacional de Agricultura Tropical (CIAT). 2005. Gridded Population of the World Version 3 (GPWv3): Population Density Grids. Palisades, NY: Socioeconomic Data and Applications Center (SEDAC), Columbia University.

GHCN station metatdata is available in the v2.temperature.inv file available from the GHCN ftp server: ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/

Here I have taken the station id, station name, latitude, longitude, and altitude from the GHCN metadata file and added a new column for population density from the GPW data. I used the adjusted data sets for the year 2000. The GPW coordinates are based on geographic coordinates of decimal degrees based on the World Geodetic System spheroid of 1984 (WGS84). I have not yet investigated the GHCN coordinate system.

There were 7280 unique records in the v2.temperature.inv file.

The initial run against the 2.5′ gridded data was able to match 6946 stations.

The remaining 334 were then run against the 15′ gridded data. 145 stations were matched at this resolution.

The remaining 189 stations were then run against the 60′ gridded data. This provided an additional 100 matches.

Of the remaining 89 stations, 14 are identified as ships. A significant number of the others are island locations.

The merged file is available here in semi-colon delimited format.

Column names are not included: id, name, lat, long, alt, popdensity

ghcn_station_inv_popdensity.txt (7280)

The intermediate files here:

ghcn_station_inv_2.5.txt (6946)

ghcn_station_inv_2.5_na.txt (334)

ghcn_station_inv_15.txt (145)

ghcn_station_inv_15_na.txt (189)

ghcn_station_inv_60.txt (100)

ghcn_station_inv_60_na.txt (89)

Most dense population? Bombay / Cola at 50607.97 people / km ^2
20743057000 BOMBAY / COLA 18.90 72.82 11 1U 5971FLxxCO 1x-9WATER C

This also shows the affect of gridding
For the coordinates 18.90 72.82 at 2.5′ resolution: 50607.97 people / km ^2
For the coordinates 18.90 72.82 at 15′ resolution: 22378.56 people / km ^2
For the coordinates 18.90 72.82 at 60′ resolution: 4773.45 people / km ^2

ghcn population density lt 1000

ghcn population density gt 1000

ghcn population density log

  1. carrot eater
    2010 March 3 at 11:39 am

    Can we see a scatter plot of the population density vs satellite nightlight brightness?

  2. carrot eater
    2010 March 3 at 11:45 am

    or also against the population number in v2.temp.inv.

    and by the way, before I go begging you to do stuff, good work and thank you.

    The strong change in result with grid density shows that this work requires that the listed coordinates of the stations are accurate.

  3. 2010 March 3 at 1:59 pm

    No probs, ce. As to ‘scatter plotting’ density-v-brightness, there are only three levels of brightness in the GHCN v2.temperature.inv. It would be more of a ‘binning.’ But I’m probably going to leave this where it is and move on to the GRUMP data this evening or tomorrow. GRUMP comes with a pop density mapping and predefined rural/urban grid.

    Anyone know where I can get a GIS mapping of ‘brightness’? Independent of GHCN?

  4. 2010 March 3 at 9:36 pm

    Hrm, is there satellite nightlight brightness data hiding somewhere in the GHCN metadata? I can’t seek to find any reference to it in ftp://ftp.ncdc.noaa.gov/pub/data/ghcn/v2/v2.read.inv.f

  5. 2010 March 3 at 9:51 pm

    Also, any chance you could run a similar analysis with all the USHCN stations in ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/ushcn-v2-stations.txt ? Thanks!

  6. 2010 March 3 at 10:17 pm

    name(31:31)=li(102:102) !US-brightness index 1/2/3=dark/dim/brite
    name(32:32)=li(68:68) !population index (R/S/U=rural/other)
    name(33:33)=li(101:101) !GHCN-brightness index A/B/C=dark/dim/brt
    name(34:36)=li(1:3) !country code (425=US)

    text_to_binary.f : code module in GISTEMP
    So column 101 in GHCN v2.temperature.inv

    Also a quick note:
    January 16,2010:The urban adjustment, previously based on satellite-observed nightlight radiance in the contiguous United States and population in the rest of the world (Hansen et al., 2001), is now based on nightlight radiances everywhere, as described in an upcoming publication. The effect on the global temperature trend is small, that change reduces it by about 0.005 °C per century.

    Yes to USHCN request.

  7. 2010 March 3 at 10:44 pm

    Thanks! Know where the US-brightness index file might be found? I presume its for USHCN.

    Here is a quick v2.mean raw run by nightlights:

  8. 2010 March 4 at 12:51 am

    Just eyeballing, it looks like there is a ~0.05C bump on the bright side over the last 30 or so years.

    Try running a diff over just the last 40 years between dark and bright and expand the y-axis a bit.

  9. 2010 March 4 at 1:35 am

    0.052 is the average since 1975. 0.173 degrees C per decade since 1975 for dark, 0.192 for bright. 0.175 for all rural, 0.172 for all rural and dark. There is remarkably little difference between using the GHCN rural designation and the nightlight designation, at least globally.

  10. 2010 March 4 at 1:39 am

    Ack, ignore those numbers, that was the slope since the early 50s. From 1975 we get:

    Dark 0.254
    Bright 0.281
    Rural 0.245
    Grim (Dark and Rural) 0.255

  11. 2010 March 4 at 4:56 am

    Total stations ushcn: 1218
    Number resolved at 2.5′: 1214
    Number resolved at 15′: 2
    Number resolved at 60′: 2
    Number unresolved: 0

    In column form (popdensity added to last col)

    If you downloaded a version of this Wed night, do it again.
    There was a problem with the first version posted.

    “Grim” => LOL

  12. 2010 March 4 at 7:37 am

    There were a couple brightness figures in the files last I looked.

    One was a brightness index..I could never find the reference for it

    I suspect that the index might have been used on Peterson2003 ( maybe owen? as a source)

    nightlights was seen by hansen as a proxy for pop density.. based on owen or gallo, cant recall off the top of my head.

    Also check the ISA products and vegatative indices

  13. 2010 March 4 at 7:41 am

    Is the gridding effect uniform? that is can you predict the density at 2.5 from the density at the higher gridding?

  14. 2010 March 4 at 7:45 am

    arrg. wasnt supposed to be a reply..

    Anyways, very cool. Would be fun to look at warming trend by density

  15. 2010 March 4 at 3:50 pm

    Thats the plan. Density, nightlights, and urban/rural designation for both GHCN and USHCN, to see if any of them change the overall trend.

    Ron, I’m not sure if you caught my earlier question about the

    name(31:31)=li(102:102) !US-brightness index 1/2/3=dark/dim/brite

    line. Since GHCN v2.temperature.inv doesn’t have a character at 102, I presume this is something for the USHCN data?

  16. 2010 March 4 at 5:35 pm

    Huh, that was unexpected: http://i81.photobucket.com/albums/j237/hausfath/Picture156.png

    Slope 1975-2009
    v2.mean all : 0.262
    v2.mean > 150 density : 0.259
    v2.mean > 1000 density : 0.271

    There are probably some spatial biases sneaking in, though the sample size of both 150 density are in the 1300 station range. I’ll see if I can rerun the analysis only for the 5×5 grids covered by v2.mean < 10 in, say, 2008 for a better comparison. Still, this is suggestive…

    rb: edit for >

  17. 2010 March 4 at 5:36 pm

    Ack, apparently less than signs cause the comment to think its HTML and eat stuff… That was:

    Slope 1975-2009 (C per decade)
    v2.mean all : 0.262
    v2.mean under 10 density : 0.284
    v2.mean over 150 density : 0.259
    v2.mean over 1000 density : 0.271

  18. 2010 March 4 at 6:29 pm

    use “&gt ;” without the space >
    use “&lt ;” without the space <

    I’d like to give commentators more control over editing their comments. I like lucia’s 10 minute edit thingee. Anyone know how to enable that in WordPress?

  19. 2010 March 4 at 6:57 pm


    Would it be possible to replicate your analysis for the 1990 and 2005 population density sets? I’d be interested to look at the effect of changing populations at station locations, and 15 years is probably long enough to get some good numbers.

    I’m envisioning comparing the temp trend in stations with ~0 population growth to stations with a fair amount of population growth for each 5×5 grid cell.

  20. 2010 March 4 at 7:30 pm

    Should be. Take a look at the nightlights threads on CA


    In the old file there were 3 data feilds:

    Nightlights ( used by GISS)
    Lights ( from GHCN)
    Brightness index:


    SteveMc used to have the file, but in the recent move it has been moved.

    My collation of GISS stations with lat longs and other info is at http://data.climateaudit.org/data/giss/giss.info.dat

  21. 2010 March 4 at 7:49 pm

    I’ve been thinking along the same lines.
    Scatter plot of 20 year trends dPopDen-v-dTemp

    I’ll produce modified v2.mean.inventory and ushcn-v2-stations.txt files with separate columns for 1990, 1995, 2000, 2005, 2010 gpw3 data with an additional column for grump Rural/Urban definitions. That way, anyone can parse the data as they see fit. Probably be Saturday before I get there.

  22. 2010 March 4 at 1:35 pm

    How “real” is the 2010 gpw3 data? I adk because I notice that they also have 2015 gpw3 population density data available as well.

  23. 2010 March 4 at 1:40 pm

    Also, since dPopDen v dTemp alone would have a spurious correlation since both are increasing.

    dPopDen v (dTemp at station – dTemp at average of stations in grid with near 0 dPopDen) would do a better job of teasing out the population-growth-related component of the temp trend.

  24. 2010 March 4 at 1:46 pm

    Also, since there is a systemic difference between a city and a rural area gaining x population, it might be useful to run the analysis both for all stations and for various subsets of stations bracketed by starting pop density (e.g. 0-10, 10-50, 50-100, etc.)

  25. 2010 March 8 at 7:52 am

    Ah yes, GPW is based on the “World Geodetic System spheroid of 1984”. That amused me when I read it (in the GPW documentation). It’s almost certainly irrelevant, and wrong. It’s irrelevant because any reasonable choice of global coordinate system will only results in shifts of up to about 120 m (and a 2.5 minute grid cell is about 3Km wide at the equator). It’s probably wrong because surely each national census reports in whatever quaint local system it pleases and the GPW people just assume it’s okay to use the coordinates as-is. For example, the UK is clearly divided into counties, have they really found polygons for UK counties expressed in WGS 84 coordinates to do the gridding? Or did they just use the UK National Grid and just assume the lat/long coords were the same? Converting all data to WGS84 would require a separate transformation for each country (or more, France uses 3 different Lambert projections for national mapping).

  1. 2010 December 1 at 10:53 pm
Comments are closed.