Refer to EBMv5_G2008_Regions_min_1ha_111-522.README for a more readable description.

# README file about the generation of the admin boundary layer

# used in the LEAC cube for Corine Land Cover (CLC) v15 data

#

# Hermann, March 2012

================================

0. Input data and GIS tools used

================================

A) Data

EuroBoundaryMap (full European coverage) - version 5.0, Jan. 2011 (short name: EBM_v5)

https://sdi.eea.europa.eu/catalogue/?uuid=8cee3968-f302-48cc-a886-3f7d6ac36a7d

Global Administrative Unit Layers (GAUL), national level (level 0) for reference year 2008 - version 2009, Dec. 2008 (short name: GAUL2008/GAUL0)

https://sdi.eea.europa.eu/catalogue/?uuid=074b8e76-5e49-467c-aa30-ed013afd41a9

Global Administrative Unit Layers (GAUL), first level (level 1) for reference year 2008 - version 2009, Dec. 2008 (short name: GAUL2008/GAUL1)

https://sdi.eea.europa.eu/catalogue/?uuid=5f9a77a6-8f6f-4551-a158-82e138336a6f

Corine Land Cover 1990 (raster 100m) - version 15, Aug. 2011

https://sdi.eea.europa.eu/catalogue/?uuid=4ead9faf-5732-4dd2-aaf1-4b10db063fe9

Corine Land Cover 2000 (raster 100m) - version 15, Aug. 2011

https://sdi.eea.europa.eu/catalogue/?uuid=38068464-b303-4221-a7e5-fec4473c3267

Corine Land Cover 2006 (raster 100m) - version 15, Aug. 2011

https://sdi.eea.europa.eu/catalogue/?uuid=87a1e572-e9fa-4090-803a-eb576ddff644

B) GIS tools

ogr2ogr and gdal_rasterize, from the GDAL - Geospatial Data Abstraction Library, http://gdal.org

Source code for version 1.9.0dev taken from trunk ( http://trac.osgeo.org/gdal/browser/trunk)

GRASS - Geographic Resources Analysis Support System, http://grass.osgeo.org/

Source code for version 6.4.2svn taken from trunk ( http://trac.osgeo.org/grass/browser/grass/trunk)

All data processing was carried out on a Debian Linux 64-bit system (EEA server name: whitefish)

Command line tools used for executing scripts: bash 4.1.5 and gawk 4.0.0

==============================================================================

1. Selection and rasterization of EBM_v5 and GAUL features into GeoTIFF format

==============================================================================

The chosen approach is a quite detailed selection of individual features from the input vector layers.

The purpose is to give different raster values to different feature types, so that later, the rasterised

features can be patched together and potentially re-classified as needed (also for non-LEAC use cases).

A) Selection of features from EBM_v5, re-projection to LAEA (EPSG:3035)

# Vector 1: NUTS3 regions from NUTS_3, where country code (ICC) is not NULL, add RID = FID + 1000

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_NUTS_3.shp StatisticalRegions/NUTS_3.shp -lco ENCODING=UTF-8 \

-sql "select ICC,NUTS_CODE,NUTS_LABEL,cast(FID+1000 as integer(8)) as RID from NUTS_3 where ICC is not NULL"

# Vector 2: Inland water from EBM_A, TAA = 7; left join EBM_NUTS.dbf to get the NUTS3 code

# create a tmp file first, which will be joined with Vector 1, in order to get the RID for the NUTS3 region

ogr2ogr -t_srs epsg:3035 $OUTDIR/tmp_EBMv5_EBM_A_TAA-EQ7.shp EBM_A.shp -lco ENCODING=UTF-8 \

-sql "select EBM_A.ICC as ICC,EBM_A.SHN as SHN,TAA,NUTS1,NUTS2,NUTS3 \

from EBM_A left join 'EBM_NUTS.dbf'.EBM_NUTS on EBM_A.SHN = EBM_NUTS.SHN where TAA = 7"

# Generate final Vector 2 shapefile, including RID

ogr2ogr $OUTDIR/EBMv5_EBM_A_TAA-EQ7.shp $OUTDIR/tmp_EBMv5_EBM_A_TAA-EQ7.shp -lco ENCODING=UTF-8 \

-sql "select E.*,N.NUTS_CODE,N.RID from 'tmp_EBMv5_EBM_A_TAA-EQ7' E \

left join '$OUTDIR/EBMv5_NUTS_3.dbf'.EBMv5_NUTS_3 N on E.NUTS3 = N.NUTS_CODE"

# Vector 3a: FullEurope, TAA = 1,3,4,7

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_TAA-LT5orEQ7.shp EBM_A.shp -lco ENCODING=UTF-8 \

-sql "select SHN,TAA,ICC from EBM_A where TAA < 5 or TAA = 7"

# Vector 3b: Add Kosovo (UNSCR 1244/99) and Serbia, TAA = 1,3,4,7

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_TAA-LT5orEQ7.shp EBM_A.shp \

-sql "select SHN,TAA,ICC from EBM_A where TAA < 5 or TAA = 7" \

-update -append # -nln EBMv5_EBM_A_TAA-LT5orEQ7

# Vector 4a: Administrative Units for Kosovo

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_AdministrativeUnits.shp AdministrativeUnits/AdministrativeUnit_2.shp -lco ENCODING=UTF-8 \

-sql "select ICC,SHN,ISN,NAMN,DESN,cast(FID+3000 as integer(8)) as RID from AdministrativeUnit_2 where ICC = 'KS'"

# Vector 4b: Add Administrative Units for Serbia

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_AdministrativeUnits.shp AdministrativeUnits/AdministrativeUnit_3.shp \

-sql "select ICC,SHN,ISN,NAMN,DESN,cast(FID+4000 as integer(8)) as RID from AdministrativeUnit_3 where ICC = 'RS'" \

-update -append # -nln EBMv5_EBM_A_AdministrativeUnits

# Vector 4c: Add Administrative Units for Moldova

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_AdministrativeUnits.shp AdministrativeUnits/AdministrativeUnit_2.shp \

-sql "select ICC,SHN,ISN,NAMN,DESN,cast(FID+5000 as integer(8)) as RID from AdministrativeUnit_2 where ICC = 'MD'" \

-update -append # -nln EBMv5_EBM_A_AdministrativeUnits

# Vector 4d: Add Administrative Units for Ukraine

ogr2ogr -t_srs epsg:3035 $OUTDIR/EBMv5_EBM_A_AdministrativeUnits.shp AdministrativeUnits/AdministrativeUnit_2.shp \

-sql "select ICC,SHN,ISN,NAMN,DESN,cast(FID+5000 as integer(8)) as RID from AdministrativeUnit_2 where ICC = 'UA'" \

-update -append # -nln EBMv5_EBM_A_AdministrativeUnits

B) Rasterisation of selected EBM_v5 features, in 1km resolution

# Raster 1: NUTS3 regions by RID

gdal_rasterize EBMv5_NUTS_3.shp EBMv5_NUTS_3_NUTS3.tif \

-tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a RID -a_nodata 0 -co compress=deflate -co tiled=yes

# Raster 2: inland water areas, by N.RID = RID of their NUTS3 region

gdal_rasterize EBMv5_EBM_A_TAA-EQ7.shp EBMv5_EBM_A_TAA-EQ7_NUTS3.tif \

-tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a N.RID -a_nodata 0 -co compress=deflate -co tiled=yes \

# Raster 3: inland water areas, by country code (8000..8999)

# a) Initialise GeoTIFF, by burning 1st inland water country code

gdal_rasterize EBMv5_EBM_A_TAA-EQ7.shp EBMv5_EBM_A_TAA-EQ7_ICC.tif \

-tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a_nodata 0 -co compress=deflate -co tiled=yes \

-burn 8003 -where "E.ICC = 'ND'"

# b) Burn remaining inland water country codes into the same GeoTIFF

awk 'NR > 4 { print $1, $2 - 1000 }' $SCRIPTDIR/EBMv5_ICC_plus_9000.txt |

while read code num ; do

gdal_rasterize EBMv5_EBM_A_TAA-EQ7.shp EBMv5_EBM_A_TAA-EQ7_ICC.tif \

-burn $num -where "E.ICC = '$code'"

done

# Raster 4: main and inland water areas, by country code (9000..9999)

# a) Initialise GeoTIFF, by burning 1st country code

gdal_rasterize EBMv5_EBM_A_TAA-LT5orEQ7.shp EBMv5_EBM_A_TAA-LT5orEQ7_ICC.tif \

-tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a_nodata 0 -co compress=deflate -co tiled=yes \

-burn 9003 -where "ICC = 'ND'"

# b) Burn remaining country codes into the same GeoTIFF

awk 'NR > 4 { print $1, $2 }' $SCRIPTDIR/EBMv5_ICC_plus_9000.txt |

while read code num ; do

gdal_rasterize EBMv5_EBM_A_TAA-LT5orEQ7.shp EBMv5_EBM_A_TAA-LT5orEQ7_ICC.tif \

-burn $num -where "ICC = '$code'"

done

# Raster 5: Administrative Units regions by RID

gdal_rasterize EBMv5_EBM_A_AdministrativeUnits.shp EBMv5_EBM_A_AdministrativeUnits_AU.tif \

-tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a RID -a_nodata 0 -co compress=deflate -co tiled=yes

C) Selection of features from GAUL2008, re-projection to LAEA (EPSG:3035)

# GAUL2008/GAUL1: select all areas which are not disputed

ogr2ogr -s_srs epsg:4326 -t_srs epsg:3035 g2008_1.shp /home/peifer/gaul/unzipped/g2008_1.shp \

-sql "select g1.*,ia.*,cast(ADM1_CODE+10000 as integer(8)) as RID from g2008_1 g1 \

left join 'international_country_attributes.dbf'.international_country_attributes ia on g1.ADM0_CODE = ia.ADM0_CODE \

where g1.DISP_AREA is NULL" -lco ENCODING=UTF-8 -overwrite

# GAUL2008/GAUL0: select all areas which are not disputed

ogr2ogr -s_srs epsg:4326 -t_srs epsg:3035 g2008_0.shp /home/peifer/gaul/unzipped/g2008_0.shp \

-sql "select g0.*,ia.*,cast(ADM0_CODE+100000 as integer(8)) as RID from g2008_0 g0 \

left join 'international_country_attributes.dbf'.international_country_attributes ia on g0.ADM0_CODE = ia.ADM0_CODE \

where g0.DISP_AREA is NULL" -lco ENCODING=UTF-8 -overwrite

D) Rasterisation of selected GAUL features, in 1km resolution

# GAUL2008/GAUL1

gdal_rasterize g2008_1.shp g2008_1.tif -tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a_nodata 0 -co compress=deflate -co tiled=yes -a RID

# GAUL2008/GAUL0

gdal_rasterize g2008_0.shp g2008_0.tif -tr 1000 1000 -te 1500000 900000 7400000 5500000 \

-ot int32 -a_nodata 0 -co compress=deflate -co tiled=yes -a RID

===================================================================================

2. Import of generated GeoTIFFs into GRASS database, growth of GAUL2008/GAUL0 layer

===================================================================================

A) Import of all GeoTIFFs from step 1 into GRASS

# Start GRASS, change to the directory where the GeoTIFFs are

for f in *.tif ; do

r.in.gdal $f out=${f%.tif} --o

done

B) Growth of GAUL2008/GAUL0 layer

# Generate GAUL0_PLUS1KM

r.grow g2008_0 out=GAUL0_PLUS1KM --o

# Let GAUL0_PLUS1KM grow..

for i in {1..184} ; do

echo "GAUL0 PLUS $i KM +++ $(date)" && echo &&

r.grow GAUL0_PLUS${i}KM out=GAUL0_PLUS$(( $i + 1 ))KM --o

g.remove GAUL0_PLUS${i}KM

done

===================================================================================

3. Patch imported raster layers together, generate final products for the LEAC cube

===================================================================================

A) Generate a patchwork of raster layers, as admin boundary layer for the LEAC cube

r.patch input=EBMv5_NUTS_3_NUTS3,EBMv5_EBM_A_TAA-EQ7_NUTS3,EBMv5_EBM_A_TAA-EQ7_ICC,\

EBMv5_EBM_A_AdministrativeUnits_AU,EBMv5_EBM_A_TAA-LT5orEQ7_ICC,\

g2008_1,GAUL0_PLUS185KM output=EBMv5.G2008.Regions_min_1ha_111-522

B) Apply a MASK to cut the generated patchwork back to the extent of CLC version 15 raster data (100m)

# The MASK itself is in 1000m resolution and covers all areas that have at least 1 ha class 111..522 data

# in the earlier generated LEAC layers: FK90, FK00 or clc06 (FK90 and FK00 layers are documented separately)

r.mask rast=laea_grid.FK90_FK00_clc06.min_1ha_111-522

C) Generate sorted tabular output for the LEAC cube

r.stats --q -c laea_grid,EBMv5.G2008.Regions_min_1ha_111-522 | awk -f filter.awk |

sort > laea_grid_EBMv5_G2008_Regions_min_1ha_111-522.sorted

The result is a text file with 5942628 rows and 3 columns. The first 10 records look like this:

GridNum GridCode ADM_ID

115471032 1kmE1547N1032 12720

115471033 1kmE1547N1033 1076

115471034 1kmE1547N1034 12720

115481031 1kmE1548N1031 1076

115481032 1kmE1548N1032 1076

115481033 1kmE1548N1033 1076

115481034 1kmE1548N1034 1076

115481035 1kmE1548N1035 12720

115481036 1kmE1548N1036 12720

Codes and names for all ADM_ID values, and the parent/child relationship of all regions,

as used in the LEAC cube is documented in: EBMv5_G2008_Regions_min_1ha_111_522_legend_final

D) Generate GeoTIFF output for other purposes

r.out.gdal EBMv5.G2008.Regions_min_1ha_111-522 createopt=compress=deflate,tiled=yes \

nodata=0 out=EBMv5_G2008_Regions_min_1ha_111-522.tif

*** END OF DOCUMENTATION ***