What is the cause for the degradation of environment?
Capitalism, corruption, consuming society? - OVERPOPULATION!
Please, save the Planet - kill yourself...

Friday, November 9, 2012

Utility for Polygon Width Calculation Is Updated

I fixed an issue of my utility for polygon width calculation in a given direction caused by redundant geometry created in certain circumstances.  'Fixed' - somewhat a too strong claim for a nasty about 100 lines long workaround-function, where I had to comment almost every line to trace what is going on (optimisation is left for the feature). Nevertheless full functionality of the utility is now available! Updated version is here.

Another note: check validity of your polygons geometries beforehand to obtain correct results.

Sunday, October 21, 2012

Fire Dynamics In Leningrad Region in 2006

Today I decided to play with TimeManager plugin for QGIS that was introduced by underdark some time ago. After some search across my spatial storage I found only one dataset that contained timestamps: FIRMS fire data that I used for creation of a methodology for burn-out probability calculation (for illegal dumping environmental risk assessment) and for assessment of human influence on fire starting.


Seems that TimeManager is not quite stable after the certain number of features in layer. At least on my machine it crushed several times (during slider manipulation) on the whole dataset of about 7000 points but worked just fine with its one-year subset of about 2700 points. But anyway overall impression is very good.

The main issue was to create a video from the generated .png files. I used console ffmpeg utils. Simple command:
:~>  ffmpeg -r 1/1 -i frame%03d.PNG -vcodec yuv420p -video_size 1126x560 output.flv
produced a 3-minute video above. One second corresponds to 1 day of the most  flammable year for the Leninngrad region in a decade. Video covers period from April to November 2006. I was too lazy to create custom undercover and just loaded OSM via OpenLayers plugin)))

Tuesday, October 2, 2012

Euthanasia Via Islam Insulting

It is a well known issue for certain group of people to get a permission get themselves killed. There are a lot of obstacles for euthanasia. And sometimes it is easier to die because of exhaustion after a loss of the process for the death permission.

But recent Islamic madness suddenly provides a "life hack" to get yourself "euthanised" quick (but dirty). All you need are internet access, the Koran  and some fire. Smart ones already got the idea, here is explanation for the rest.



The basic idea is old as the religions existence: if you don't pray %god's name% - you must die the most violent way. Nowdays it is [so called] Allah's turn to punish faithless. So if you are in need of euthanasia and don't want to wait for the legal permission to die, and for some reasons unable to kill yourself - just burn any book (any that you can grab - but statistically chances are it actually will be the Koran or the Bible), spread the word that you have burned the Koran (post video on YouTube, Vimeo, post pictures in every social network you can register and send all the multimedia with "I hate you letter" to the local Muslim community, etc.). Make Islamic world aware that you don't belong to it. Ah, and don't forget to provide your address! Alternatively (if you are paralysed) you may just mention that you don't like Muhammed too much instead of burning books. This will do too. 

Now you can relax and make the last thing you wanted to - your days are numbered! As a bonus if you'll get extremely lucky, they will even place a bounty on your head! Just try not to laugh too loud at their faces when they will come to kill you - it's a sin bad manner to laugh at alternatively gifted.

Friday, September 21, 2012

Utility to Calculate Polygon Width in Any Given Direction

I have created an "Azimuth-Width" utility recently. It is designed to compute (for now) maximum or minimum width of the given polygon(s) in the given direction. Corresponding QGIS plugin coming soon (or not too soon), so this utility may help these who are in need right now.


Prereqirements:

  1. You have to have QGIS installed on your machine.
  2. QGIS have to be in PYTHONPATH. See: http://qgis.org/pyqgis-cookbook/intro.html
  3. If you are non-Windows user ensure that command "QgsApplication.setPrefixPath( qgis_prefix, True)" in the utility file have  a valid path to QGIS installation. If not - modify "qgis_prefix" to set correct value. It is "/usr" now - must work in most of cases.

General usage:

Copy width.py to the directory with a shh-file containing polygons.

Using console navigate to the directory. In console type:
:~> python ./width.py [file to analyse] [field to store values] [azimuth (decimal degrees)] [mode ('min' or 'max')] [mode-2 ('abs', or 'rel')] [algorithm ('byStep' or 'byVertex' or 'Mix')] [step (real numver, CRS units; for 'byStep' and 'Mix' only)]

Example:~> python ./width.py poly.shp width 285.9 max abs byStep 1.3

Where:
  • ./width.py - name of this utility file.
  • [file to analyse] - a shp-file with polygons to analyse.
  • [field to store values] - if it does not exist it will be created.
  • [azimuth] - a direction for width calculation. Accepts decimal degrees from 0 to 360.
  • [mode] - type of width to calculate. Currently 'min' (returns minimum width value in given direction) and 'max' (returns maximum value in given direction) modes are available. Mode 'min' used with 'byVertex' algorithm will return minimum polygon width different then 0.0. This will be different (greatly most time) to 'min' mode and 'byStep' or 'Mix' algorithm.
  • [mode-2] - if polygon is not convex it may have several segments in given direction. If you want to take sum of the segments of the result - use 'abs', if you want only the longest (shortest) segment - use 'rel'. Fixed: Note that currently 'rel' sometimes will provide incorrect results for non-convex polygons and for convex polygons with redundant vertices on its edges. This issue will be solved when the behaviour of the intersection() command will be changed or when I will implement a workaround for it.
  • [algorithm] - algorithm that will be used: 'byStep', 'byVertex' or 'Mix'. "byStep" algorithm (improved version of the general idea that described here) will take provided value (shp-file CRS units) and swipe the polygons line with the given step along the bounding box. Precision depends on the step. Speed and precision depends on step - lower step mean more precise result but computation will take more time. "byVertex" (general idea described here) will intersect polygon with the line only in vertexes of the given polygon. For convex polygons this algorithm will be faster and more precise using 'max' mode then "byStep" algorithm. But this algorithm is not suitable for 'min' mode. "Mix" algorithm will use both "byVertex" and "byStep" algorithm so in some cases it will be most precise but will consume even more time than 'byStep'.
  • [step] - step for "byStep" and "Mix" algorithm. Takes decimal values in shp-file CRS units. Lower step means more precision but more computation time.

Some advises:

  1. Use Equal Area projections.
  2. Make sure your polygons have correct geometry. Otherwise the calculations will be incorrect.
  3. 'byStep' algorithm should be faster when polygon have enormous number of vertices.
  4. If you have a large variety in polygons area e.g. a continent and an island to save computation time you may define big step (like 1000 m or so) to swipe through continent faster. It will save computation time and a width for the small island will be calculated even if step is grater then any side of island's' Bounding Box: step for the island in this case will be 1/100 of the shortest side of it's Bounding Box.

Sunday, September 2, 2012

Raster Extent Modification Using QGIS

UPDATE: you can download script for Processing module to perform this task from here.

When you need to perform algebraic operations with two or more rasters you often have to ensure that these rasters have the same extent, otherwise the operation either  won't be allowed (in SAGA for example) or you will get output raster cropped to the overlapping part. With regular (not georeferenced) images you can simply load them in GIMP and increase or decrease canvas for amount of pixels you like. But in case of georeferenced images you will loose spatial reference information. 

In QGIS you can change extent of the rasters. Lets examine one of the worst case scenarios. There are two overlapping (one band) rasters A and B. Say, we need to add A values to B values and get the final image to have extent that will contain both images.

Overlapping rasters
We will use QGIS's Raster calculator. Firstly we need to create an empty georeferenced image with needed extent. Go to Raster -> Miscellaneous -> Build Virtual Raster. Here choose rasters A and B to become a virtual raster (VR) and do not define noData values. Add VR to the TOC.

Now we will create a templates for expanded A and B. In Raster calculator (Raster -> Raster calculator) insert expression: VR@1 * 0, highlight VR raster (by clicking on it) in Raster bands and click on Current layer extent button. Execute expression and save its result - it will be a template for raster A. Copy it to create template for raster B.

Now lets change extent for A. In Raster calculator enter expression:  A@1 this will simply leave A values unchanged. Then click on VR raster in Raster bands to highlight it and click a on Current layer extent button - this will set desired extent of the output raster. Save result as a template for A (overwrite existing file). Do the same for B.

Now you have both A and B with the same extent and can freely perform calculations over them.

Monday, August 13, 2012

Rgdal Package: R crash during "writeGDAL" solved

Lost entire day fighting this ridicules crash during export of raster created in R to GeoTiff (other "rgdal" functions worked fine). This option worked well several months ago when I last time used it, so I wondered what happened. I searched for the only error message I was able to catch: "Failure during IO", but wasn't able to find anything useful. 

After some time I started to suspect GDAL itself in abnormal behaviour since its latest updates caused some troubles already. I was wrong but the idea itself turned to be useful - I recalled that I had about a dozen of GDAL updates since "rgdal" package was installed. Despite the package was "up to date" I've reinstalled it. It solved the problem)))

Wednesday, August 1, 2012

Sverdlovskaya Region, Russia: a Field Survey in Autumn. Part 2: Scars on the Earth

Ok, it's time to finish the story about land monitoring in Sverdlovskaya region. In this post I would like to demonstrate some of the most unpleasant types of the land use.

Lets begin with illegal dumping. This dump (note that there is the smoke from waste burning down) is located right next to the potato field (mmm... seems these  potato are tasty). The ground was intentionally excavated here for dumping waste. Obviously this dump is exploited by the agricultural firm - owner of this land, but who cares...
Panorama of freshly burnt illegal dump

The next stop is peat cutting. A huge biotops are destroyed for no good reason (I can't agree that use of peat as an energy source is a good one). At the picture below you can see a peat cutting with the area of 1402 ha. There are dozen of them in the study area...
Peat Cutting (RapidEye, natural colours)
But the most ugly scars on the Earth surface are left from mining works. There is Asbestos town in Sverdlovskaya region. It was names after asbestos that is mined  there. The quarry has an area of 1470 ha and its depth is over 400 meters. Its slag-heaps covers another 2500 ha... The irony is that this quarry gives a job for this town and killing it. You see, if you wand to dig dipper you have to make quarry wider accordingly. Current depth is 450 m and in projects it is over 900 m, but the quarry is already next to the living buildings. So quarry is going to consume the town... By the way, the local cemetery was already consumed. Guess what happened to human remains? Well, it is Russia, so they were dumped into the nearest slug-heap.

Here is the panorama of the quarry. You may try to locate BelAZ trucks down there ;-)
Asbestos quarry
Here is the part of the biggest slag-heap:
A slag-heap
That's how it looks from space:
Asbestos town area (imagery - RapidEye, NIR-G-B pseudo-colour composition)
And in the end I will show you the very basic schema of disturbed land in the study area (no settlements or roads included). Terrifying isn't it? 

Basic schema of disturbed land 

Wednesday, June 27, 2012

Conference Announcement: Open GIS!

Open GIS! is a conference for users and developers of open-source GIS. It will take place on November 17-18, 2012 in Moscow, Russia. The really cool thing is that participation in this crowd-source conference is free of charge!!! Of course modest donations will be appreciated and a sponsorship as well, but it is not mandatory - we want this event to be available for everyone who are interested in open geospatial software and data. Download leaflet!

A call for papers will be opened in August but already now you may propose topics of your talk or master-class to the committee.

Please, spread the word!

P.S. If you are (a rear) person who interested in illegal dumping research (from geospatial point of view) - come to this conference to hear my talk on it!

Wednesday, June 20, 2012

How To Get Rid of Needless SRTM Files Fast

It is quite annoying to download the whole 1Gb archive of SRTM30 DEMs when you need only couple of dozens of them. In case of Leningrad region for example I had to download 4 archives and face necessity of finding needed DEMs amongst several thousands(!!!) of files and deletion the rest... Too tedious and too long task to do it manually. Lets optimise it!

We will use QGIS and Linux (I prefer openSUSE) console (and a text editor of your choice). The algorithm is applicable to other similar issues too.

Firstly we will get the list of names of files that we need to save (for our actual project). Collect all DEMs from archives in a single folder. And use Image Boundary plugin (or Raster tile index from "Raster" menu) to create a polygon shp-file with borders of all the DEMs from the folder and corresponding file name and path.

Open this layer in your canvas and select polygons that intersects your area of interest:


Now you can either to "save selection as" CSV from the layer menu or "copy selected rows to clipboard" from attribute table and paste it to the empty text-file. In LibreOffice Calc (or use other text editor for this) delete all columns exept that which contains file names and add to the list name of the text-file itself (lets name it FILELIST). NOTE: this file should contain ONLY NAMES without paths. Place this file to the directory with DEMs.

We are ready for the final step - deletion of a thousands of DEMs that we are not going to use. A recipe I found here: using console go to the folder with DEMs and execute following command:
ls -1 | sort FILELIST FILELIST - | uniq -u | xargs rm
Where FILELIST is our text-file containing names of files that we need to KEEP. This command will delete all files from the folder except files which names are listed in FILELIST.

Saturday, June 16, 2012

About On-line Education

Recently I've accomplished 2 of the on-line courses provided by Coursera

A "Model Thinking" course provided a nice overview of the existing approaches to model creation. Despite oversimplified maths (that I tried to compensate by using R-scripting where it was applicable) I liked this course for covering a wide range of real world implementation of different types of models.

A "Game Theory" course was quite nice, but had a lack of the real world implementation examples (there were a couple of them). And it was quite disappointing because that was what I hoped to learn about. I've already studied Game Theory in my University 7 or 8 years ago and since then was curious in seeing the theory in action. Unfortunately all the examples I've encountered were either oversimplified or far too complicated. The good thing was that the statement of accomplishment of Game Theory course was much fancier then that of Model Thinking.

Friday, June 8, 2012

A Bit More on Ignorance

Occasionally I found an article that have some relation to my previous post. The article has an intriguing name: "The Influence of Map Design on Resource Management Decision Making". Unfortunately it is not in open access, so I wasn't able to read it. Also abstract omits conclusions... And I would prefer to see study of the real cases... Nevertheless here you are:

  Abstract
The popular use of GIS and related mapping technologies has changed approaches to map-making. Cartography is no longer the domain of experts, and the potential for poorly designed maps has increased. This trend has raised concerns that poorly designed maps might mislead decision makers. Hence, an important research question is this: Can different cartographic displays of one data set so influence decision makers as to alter a decision that relies on the mapped data? This question was studied using a spatial decision problem typical for decision makers in a resource management agency in the United States (the USDA Forest Service). Cartographic display was varied by constructing three hypothetical map treatments from the same data set. Map treatments and other test materials were mailed to Forest Service District Rangers. All District Rangers received the same decision problem, but each received only one of the three map treatments. The participants were asked to make a decision using the decision problem and map treatment. Information about the decision and the influence of each map treatment was obtained through a questionnaire. The research and its implications for map-based decision making are presented and discussed.

Monday, June 4, 2012

About Spatial Ignorance

There is one question about spatial technology that bothering me for a couple of years. 

Is anyone suitable for working with GIS and spatial data or one should be a professional?  

Well that's a tough question. But in general it depends on conditions. 

Lets ask ourselves two more other questions.
  1. Is anyone suitable to operate PC, printer or database?
  2. Is anyone suitable to be system administrator in your office?
Obviously it is "yes" for the first question and "no" for the second. Anyone can ride a bicycle but one need license to drive a bike.

So my answer for the given question is - one need to learn a lot to understand how to work with spatial data. Anyone willing to learn theory is suitable to work with spatial data.

Tonight one smart ass (or should I write "cat"?) asked me why in the hell I suggest to calculate distances (when using PostGIS) in the corresponding zone of Pulkovo-42 for Russia (EPSG:28403 for example) when possible? The motivation of the question was based his statement that one will need far shorter SQL-query for distance measurement if the geometry would by casted to geography type and distance would be calculated with EPSG:4326. Like: "Eah, ST_Distance will return meters even for ellipsoidal calculations! Did you know it, dumb ass?! Admit, you suck!"

Well, kind reader, can YOU see the GLOBAL difference between  EPSG:28403 and EPSG:4326 (it is not about projection)? The answer is "reference ellipsoid" (or "geodetic datum" or just "datum"). This reference ellipsoid defines the Earth shape model parameters for all the calculations that one would perform with spatial data.

Well, kind reader, if you were (are) a good student then you should understand, that if reference ellipsoid defines the MODEL of the shape, then it also defines the BIAS of this model from the true value at any given location.

The next questions you may ask: 

Q: Does this bias matters?
A: Well, it depends on your goal requirements and the answer will be "NO" or "YES". So you may ask me here if in doubt. 

Q: How big is the difference between measurements using different datums?
A: Well, it depends ;-) ... depends on datums compared as well as locations compared - from centimeters to hundreds of meters. 

Q: Will 1 meter difference make your day?
A: Absolutely! Do you have a land parcel in your possession? Imagine that tomorrow some people from the municipality will came to you and demand to move you fence, because it is build on the adjusted road land parcel. They show you a digital map where your fence is 1 meter on the adjusted parcel. And this shit happened because of 1 meter bias! 

Instead of conclusion

If you're a cat - grab a handbook on geodesy and become a human being, or... just keep licking your balls and quit work with spatial data before you'll ruin something.

Monday, May 14, 2012

Spatial Randomness Evaluation in R: Monte Carlo Test

This post is a some kind of reply to this one.

So our goal is to determine whether our point process is random or not. We will use R and spatstat package in particular. Spatstat provides a very handy function for this, that uses K-function combined with Monte Carlo tests. I will spear you from burbling  about theory behind it - the necessary links were already provided. Lets get directly to action.

In this example I will test data about location of my "favourite" illegal dumps in St. Petersburg and Leningrad region.

# we will need: 
library(maptools) 
library(rgdal) 
library(spatstat)

# import data for analysis

S <- readShapePoints("custom_path/dump_centroids.shp", proj4string= CRS("+proj=tmerc +lat_0=0 +lon_0=33 +k=1 +x_0=6500000 +y_0=0 +ellps=krass +towgs84=23.92,-141.27,-80.9,-0,0.35,0.82,-0.12 +units=m +no_defs"))

SP <- as(S, "SpatialPoints")
P <- as(SP, "ppp")

# perform the test itself with a 100 simulations

E <- envelope(P, Kest, nsim = 100)
plot(E, main = NULL)


And here is what we've got in the end, a fancy graph, which demonstrates that our data (Kobs) significantly deviates from a random process (Ktheo):



Monday, April 23, 2012

Illegal Dumping in Leningrad Region: Press Conference

There was a press conference on Tuesday the 19-th about illegal dumping in Leningrad region (Russia). I was asked to be the main speaker there and to present to the press my recent study on illegal dumping prevention. I've already had two presentations on this subject recently at the international scientific conference in St. Petersburg State University and at the round table for the discussion of the upcoming "Let's do it. Russia" clean up event.

Some video from the press conference:


The main conclusion that I made by investigating possible impacts on illegal dumping prevention (such as penalty increase, chance of being caught increase and waste disposal fare decrease) is that decrease of the waste disposal fare for population is the most efficient way. And I managed to find two other publications that came to the exact conclusion (for example, there is an evidence that 1% waste fare increase leads to 3% increase of illegal dumping cases).

By the way I was able to assess probability of being caught for illegal dumping in Russia. It is about 10-5 (you can die while playing soccer with such probability).

The only way to reduce waste fares is to use waste as a resource. That means that the only way to prevent illegal dumping is to create waste management system that would be able to complete the zero waste goal.

And here is an abstract from my article:

Mechanisms of the land protection were discussed in this article. An algorithm of decision making whether to dump illegally or not was explained. Formulas for determination of profitable ration of expenditures per unit and amount of illegally dumping waste are substantiated. Effect from different types of impacts that can be used for land protection from illegal dumping were discussed (such as fares change, penalties change, penalty application probability change). Decreasing of waste disposal fares was acknowledged as the most effective way for illegal dumping prevention, but it is possible only if «zero waste» concept is implemented.

Thursday, April 12, 2012

D&D: About Experience Awad

To my mind one of the most tedious part of the D&D game is an XP award calculation. DM have to spend hours and even days preparing adventure, so spending extra hours on XP award calculation seems quite expensive. Upcoming changes to the system wouldn't make a difference.

Would you really want to calculate all that CRs and ELs? Nine Hells, NO!!!

Smart people tends to save their time using own systems. This one for example is quite clever. In general - XP awarded for caused and taken points of damage. I have no intentions discuss its Pros and Cons, but would like to admit that this approach still needs calculation and statistics records. So it is cheaper (in terms of time) but still is expensive for my taste.

My system of XP award


Tuesday, April 10, 2012

MOSKitt Geo: a Tool for Spatial Database Design

I needed to create design for a spatial database and found out that despite of availability of the tools for a regular database design, there is a lack of such tools for a spatial database design.

Fortunately there is a suitable free cross-platform tool - MOSKitt Geo, which allows to design spatial databases for PostGIS and Oracle. MOSKitt is not quite user-friendly and if you are not an experienced software designer, you will need to read the f*** manual.

Installation

  • Download MOSKitt itself and extract it (installation is not required). Note that x86_64 version (and version for MAC) is available for 1.3.2 and earlier releases (current is 1.3.8).
  • Install MOSKitt Geo plugin using instructions from here. Note that for 3-rd step in instruction you may also need to type something and erase it in the search area of the dialogue to make list of plug-ins appear (at least I needed to). And one more thing - MOSKitt Geo repository is already there, so you don't need to add it - just choose it from the drop-down menu.

PostgreSQL/PostGIS database connection and reverse engineering

To connect to existing database go Window->Show View->Other->Data Management->Data Source Explorer
  
A new tab will occur:

Right click on Database Connections and choose New->PostgreSQL. You will see a dialogue for the DB connection settings. Note that there might be no PostgreSQL JBC Driver defined. In this case first download needed driver and then click on the icon to the right from the driver drop-down menu (New Driver Definition).
Connection settings

 In the Edit Driver Definition dialogue provide path to the driver and click Ok.
Driver definition dialogue
When database connection established, right-click on it and choose Reverse Engenieering.

Wednesday, April 4, 2012

My Gmail Content instead of video in YoutTube... WTF?!

A funny snapshot of what I sow instead of video on YouTube - a part of my current page from Gmail! And if I change page in Gmail, the video frame content changes respectively. This thing happened when the PC turned on from the suspend mode (but it never happened before).

This is an actual snapshot (not a photoshop) of the Gmail page instead of video

Tuesday, April 3, 2012

Universal Ecolabeling: an "eco-Black-Spot" concept

Nowadays  more and more people cares about environmental-friendly goods. Ecolabeling is often used to distinguish such goods from regular goods. 

But there are several issues:
  • there are a lot of ecolabels, provided by different organisations with different standards so quality of the given ecolabel usually is uncertain;
  • ecolabels are difficult to recognize: if you unaware about given ecolabel and know nothing about authority, that gave it - you are likely to miss the point that this is ecolabel, or even you can be tricked by companies with some green-coloured sign with a word "eco", this leads us to the following point -
  • often they can't be trusted - companies like to place their own labels that states that product is friendly to environment to attract customers, but these labels are just a trick and has nothing to do with reality.
So I would like to propose another approach. Do you remember BP re-branding after  Deepwater Horizon explosion and infamous oil spill? I hope you do. What if we will oblige companies to put negative labels on the products which production causes significant threat to environment?! - A universal "eco-Black-Spot" for environment abusers!

Pros: 
  • we won't need external sertification organisation to confirm that this company deserves an eco-Black-Spot - an environmental agencies already have the necessary information;
  • nobody will forge eco-Black-Spot for their goods;
  • forged (self-made) "ecolabels" will instantly fail being alongside with the eco-Black-Spot.
  • companies will try to wipe out an eco-Black-Spot from their products by actual contribution for preservation of the environment.
Cons:
  • m-m-m... I don't see any. Do you?
And what about design of the eco-Black-Spot? I believe it will be convenient to use something like this:
or this:


Wednesday, March 28, 2012

I'm a Father Now

On the 23 of March my wife gave a birth to a beautiful girl. We are going to name her Alice (the name that stayed last after we've cut out all other names from the list one by one). But I've not abandoned yet my insidious plan to (secretly from her mom) register her as Hatshepsut ;-)


Sunday, March 11, 2012

QGIS and GDAL>=1.9 Encoding Issue: a Workaround

After significant encoding handling changes in GDAL>=1.9 it became quite challenging to handle non-Latin attributes (Cyrillic in particular) stored in .dbf part of a shp-file. So the majority of the Cyrillic and other non-Latin users have to stick with GDAl 1.8.1 for now.

The cause of the issue is described here (Rus). Briefly: GDAL>=1.9 attemts to re-encode the .dbf-file to UTF-8 on the basis of the LDID (Language Driver ID) written in .dbf header. But unfortunately LDID is usually missing, and in particular QGIS does not write it to the .dbf-file it creates. In case when LDID is missing, GDAL>=1.9 assumes that encoding of the .dbf-file is ISO8859_1 (Latin-1) which makes non-Latin characters unreadable.

The workaround I'm currently using is creating additional .cpg-file, that contains the ID of the encoding used. For example if encoding is Windows-1251, .cpg-file contains the following record: "1251" (without quotes). When .cpg-file is present, GDAL>=1.9 + QGIS works just fine.

UPD: on some OS you will need to use ID from Additional ID column instead of Encoding ID column.

UPD2: For Windows you may also try to use "unofficial" version of QGIS from here (with encoding issue solved). But it is possible that its installer is in Russian.

UPD3: There is another workaround. You can open .dbf-file in Libre Office Calc (Open Office Calc) providing encoding needed and save it from there. This will write necessary header to .dbf-file and QGIS will open attributes correctly.  Note that this also will make fields names written in upper case.

UPD4: there is a plugin for encoding fixing available.


Here you are a table of the encoding IDs (taken from here):
 
-->
Encoding ID Encodind name Additional ID Other names
1252 Western iso-8859-1
except when 128-159 is used, use "Windows-1252"
iso8859-1, iso_8859-1, iso-8859-1, ANSI_X3.4-1968, iso-ir-6, ANSI_X3.4-1986, ISO_646, irv:1991, ISO646-US, us, IBM367, cp367, csASCII, latin1, iso_8859-1:1987, iso-ir-100, ibm819, cp819, Windows-1252
20105 us-ascii us-acii, ascii
28592 Central European (ISO) iso-8859-2 iso8859-2, iso-8859-2, iso_8859-2, latin2, iso_8859-2:1987, iso-ir-101, l2, csISOLatin2
1250 Central European (Windows) Windows-1250 Windows-1250, x-cp1250
1251 Cyrillic (Windows) Windows-1251 Windows-1251, x-cp1251
1253 Greek (Windows) Windows-1253 Windows-1253
1254 Turkish (Windows) Windows-1254 Windows-1254
932 Japanese (Shift-JIS) shift_jis shift_jis, x-sjis, ms_Kanji, csShiftJIS, x-ms-cp932
51932 Japanese (EUC) x-euc-jp Extended_UNIX_Code_Packed_Format_for_Japanese, csEUCPkdFmtJapanese, x-euc-jp, x-euc
50220 Japanese (JIS) iso-2022-jp csISO2022JP, iso-2022-jp
1257 Baltic (Windows) Windows-1257 windows-1257
950 Traditional Chinese (BIG5) big5 big5, csbig5, x-x-big5
936 Simplified Chinese (GB2312) gb2312 GB_2312-80, iso-ir-58, chinese, csISO58GB231280, csGB2312, gb2312
20866 Cyrillic (KOI8-R) koi8-r csKOI8R, koi8-r
949 Korean (KSC5601) ks_c_5601 ks_c_5601, ks_c_5601-1987, korean, csKSC56011987
1255 (logical) Hebrew (ISO-logical) Windows-1255 iso-8859-8i
1255 (visual) Hebrew (ISO-Visual) iso-8859-8 ISO-8859-8 Visual, ISO-8859-8 , ISO_8859-8, visual
862 Hebrew (DOS) dos-862 dos-862
1256 Arabic (Windows) Windows-1256 Windows-1256
720 Arabic (DOS) dos-720 dos-720
874 Thai Windows-874 Windows-874
1258 Vietnamese Windows-1258 Windows-1258
65001 Unicode UTF-8 UTF-8 UTF-8, unicode-1-1-utf-8, unicode-2-0-utf-8
65000 Unicode UTF-7 UNICODE-1-1-UTF-7 utf-7, UNICODE-1-1-UTF-7, csUnicode11UTF7, utf-7
50225 Korean (ISO) ISO-2022-KR ISO-2022-KR, csISO2022KR
52936 Simplified Chinese (HZ) HZ-GB-2312 HZ-GB-2312
28594 Baltic (ISO) iso-8869-4 ISO_8859-4:1988, iso-ir-110, ISO_8859-4, ISO-8859-4, latin4, l4, csISOLatin4
28585 Cyrillic (ISO) iso_8859-5 ISO_8859-5:1988, iso-ir-144, ISO_8859-5, ISO-8859-5, cyrillic, csISOLatinCyrillic, csISOLatin5
28597 Greek (ISO) iso-8859-7 ISO_8859-7:1987, iso-ir-126, ISO_8859-7, ISO-8859-7, ELOT_928, ECMA-118, greek, greek8, csISOLatinGreek
28599 Turkish (ISO) iso-8859-9 ISO_8859-9:1989, iso-ir-148, ISO_8859-9, ISO-8859-9, latin5, l5, csISOLatin5

Tuesday, February 21, 2012

Speeding Up QGIS Rendering - a Workaround (sort of)

It is known that QGIS is painfully slow in redrawing canvas when at least one of the vector layers contains a lot of features. The obvious way to speed up the process is to forbid to render layer (or some of the features) at certain scale(s). Yes, it works, but what do we have to do, when we want to see the whole picture? To disable scale rendering and wait couple of minuets until the several hundreds of thousands features will be redrawn? Bo-o-oring!

The workaround that I use is not that elegant (as a rule based feature scaling for example), but it can be used in certain situations. When your vector layer is large but it is necessary to see all the features - start a WMS-server for this particular layer. Raster renders a way more quickly and will be generated by different software. Taking into account that the layer is on the same machine or in the local network the speed increase will be magnificent.

So you produce WMS-layer from the needed layer (lets call it parent-layer) by WMS-sever of choice (I use GeoServer because it has a straightforward GUI that allows you to set up server and WMS-layer in minuets), add it in QGIS and place it under the parent-layer. Then set visibility of the parent-layer to the larger scale (that is suitable for editing). Now you will be able to see the whole picture at the small scale, and rendering still will be fast. And because WMS-layer is produced from the same layer all of the committed changes to the parent-layer will appear at WMS-layer immediately.

And you will have another thing to decide: whether to create the same [complicated] style for WMS-layer as for parent layer or not. Well, I do not bother)))
Small Scale
Large scale




Friday, February 17, 2012

About Environmental Risk Assessment

There was a scientific seminar dedicated to environmental risks assessment in the scientific-research centre where I work. A speaker was awfully ignorant in subject unfortunately. As a person who is experienced in environmental risk assessment (see my posts about risks and a particular methodology) I was afraid that I will be the one to ask the speaker (quite an old man) some inconvenient question about formulas he used, but luckily he was ashamed by someone else.

During the discussion the question of monetary aspect of the risk and damage to environment was raised: whether it is possible to use money as the measure of risks that only applicable to environment itself. In other words: is it rational to use money when assessing possible damage to solely ecosystem (there are no money in ecosystem by itself), and how to perform such assessment?

What do YOU think? I wasn't able to find an appropriate answer at that moment, but now I believe I have a point. My answer is YES, we can use money to assess risks and damage dealt to ecosystem only.

Firstly the assessment is made by humans and for humans. And humans understand monetised value more easily. The approach that I want to propose is about assessment of money that have to be spent to recover ecosystem to exact the same state it was prior to caused or possible damage. Just imagine how much money one have to spent for recreating and reintroduction of just one extinguished species (a tasmanian wolf for example). Here you are a monetised damage to environment.

Another approach I have in mind is about evaluation of risks via relative live value of species (which can be easily monetised too). Lets use this formula for evaluation of life of individual of a given species: V=(1/N)*P, where V - relative value, N - population of the given species (or given areal of species), P - total population of the human beings. We will have a relative value as 1 for humans and 1*(P/N) for a given species. For example for a tiger we will have its relative individual value about 1 076 900! Literally, if we have a choice whether to save 1 million people or a single tiger, the tiger must be saved - not a million of people!!!

And we can monetise this value by multiplication on the average value of the single human life (you can play a bit with numbers given here).

So the damage to ecosystem may be assessed via loss of number of individuals of species that live in a given ecosystem and we are able to easily evaluate a relative value of the individuals of the each species, and it can be easyly monetised.