Interactive Visualisation of the Profitable Amount of Waste to Dispose Illegally

"Wow!" - I said to myself after reading R Helps With Employee Churn post - "I can create interactive plots in R?!!! I have to try it out!"

I quickly came up with an idea of creating interactive plot for my simple model for assessment of the profitable ratio between the volume waste that could be illegally disposed and costs of illegal disposal [Ryabov Y. (2013) Rationale of mechanisms for the land protection from illegal dumping (an example from the St.-Petersburg and Leningrad region). Regional Researches. №1 (39), p. 49-56]. The conditions for profitable illegal dumping can be describes as follows:

Here: k - the probability of being fined for illegal disposal of waste;
P - maximum fine for illegal disposal of waste (illegal dumping);
V - volume of waste to be [illegally] disposed by the waste owner;
E - costs of illegal disposal of waste per unit;
T - official tax for waste disposal per unit.

The conditions for the profitable landfilling can be described as follows:

Here: V1 - total volume of waste that is supposed to be disposed at illegal landfill;
Tc - tax for disposal of waste at illegal landfill per unit;
P1 - maximum fine for illegal landfilling;
E1 - expenditures of the illegal landfill owner for disposal of waste per unit.

Lets plot the graphs (with some random numbers (except for fines) for a nice looking representation) to have a clue how it looks like.

Note that there is a footnote (this post provides nice examples on how to do it) with the values used for plotting - it is important to have to have this kind of indication if we want to create a series of plots.

Now I will show you the result and then will provide the code and some tips.

Playing with the plot

Tips and Tricks

Before I will show you code I want to share my hardly earned knowledge about nuances of the manipulate library. There are several ways to get static plot like that using ggplot, but some of them will fail to be interactive with manipulate.

1. All the data for the plot must be stored in one dataframe.
2. All data for plots must be derived from the dataframe (avoid passing single variables to ggplot).
3. Do not use geom_hline() for the horizontal line - generate values for this line and store them inside dataframe and draw as a regular graph.
4. To create a footnote (to know exactly which parameters were used for the current graph) use arrangeGrob() function from the gridExtra library.
5. Always use $ inside aes() settings to address columns of your dataframe if you want plots to be interactive

The Code

library(ggplot2)
library(grid)
library(gridExtra)
library(manipulate)
library(scales)
library(reshape2)

## Ta --- official tax for waste utilisation per tonne or cubic metre.
## k --- probability of getting fined for illegal dumping the waste owner (0 V, y <=> E

max_waste_volume <- 2000 
Illegal_dumping_fine_P <- 300000
Illigal_landfilling_fine_P1 <- 500000
Fine_probability_k <- 0.5
Official_tax_Ta <- 600

# mwv = max_waste_volume
# P = Illegal_dumping_fine_P
# P1 = Illigal_landfilling_fine_P1
# k = Fine_probability_k
# Ta = Official_tax_Ta

updateData <- function(mwv, k, P1, P, Ta){
    
    # creates and(or) updates global data frame to provide data for the plot
    
    new_data <<- NULL
    new_data <<- as.data.frame(0:mwv)
    names(new_data) <<- 'V'
    new_data$IlD <<- k*P1/new_data$V
    new_data$IlD_fill <<- new_data$IlD
    new_data$IlD_fill[new_data$IlD_fill > Ta] <<- NA # we don't want ribbon to fill area above Official tax 
    new_data$IlL <<- Ta-k*P/new_data$V
    
    new_data$Ta <<- Ta
    new_data$zero <<- 0
    dta <<- melt(new_data, id.vars="V", measure.vars=c("IlD", "IlL", "Ta"))
    dta.lower <<- melt(new_data, id.vars="V", measure.vars=c("IlD_fill", "zero", "Ta"))
    dta.upper <<- melt(new_data, id.vars="V", measure.vars=c("Ta", "IlL", "Ta"))
    dta <<- cbind(dta, lower=dta.lower$value, upper=dta.upper$value)
    dta$name <<- factor(NA, levels=c("Illegal landfill owner's\nprofitable ratio",
                                    "Waste owner's\nprofitable ratio", 
                                    "Official tax"))
    dta$name[dta$variable=="IlD"] <<- "Illegal landfill owner's\nprofitable ratio"
    dta$name[dta$variable=="IlL"] <<- "Waste owner's\nprofitable ratio"
    dta$name[dta$variable=="Ta"] <<- "Official tax"
}

updateLabels <- function(k, P1, P, Ta){
    
    ### creates footnote caption for the plot
    
    prob <- paste('Fining probability = ', k, sep = '')
    landfilling_fine <- paste('Illegal landfilling fine = ', P1, sep = '')
    dumping_fine <- paste('Illegal dumping fine = ', P, sep = '')
    tax <- paste('Official tax = ', Ta, sep = '')
    note <<- paste(prob, landfilling_fine, sep = '; ')
    note <<- paste(note, dumping_fine, sep = '; ')
    note <<- paste(note, tax, sep = '; ')
    note
}


plotDumping <- function(mwv, P, P1, k, Ta){
    
    ### this function draws the plot
    
   # initialise plot data
    updateData(mwv, k, P1, P, Ta)
    updateLabels(k, P1, P, Ta)
    
    # draw the plot
    profit <- ggplot(dta, aes(x=dta$V, y=dta$value, ymin=dta$lower, ymax=dta$upper, 
                              color=dta$name, fill=dta$name, linetype=dta$name)) +
        geom_line(size=1.2) + 
        geom_ribbon(alpha=.25, linetype=0) +
        theme(axis.text.x = element_text(angle=0, hjust = 0),
              axis.title = element_text(face = 'bold', size = 14),
              title = element_text(face = 'bold', size = 16),
              legend.position = 'right',
              legend.title = element_blank(),
              legend.text = element_text(size = 12),
              legend.key.width = unit(2, 'cm'),
              legend.key.height = unit(1.2, 'cm'))+
        scale_linetype_manual(values=c(4, 5, 1)) +
        scale_fill_manual(values = c("#F8766D","#00BFC4",NA)) +
        scale_color_manual(values = c("#F8766D","#00BFC4", '#66CD00')) + 
        labs(title="Profitable ratio between the volume \nof illegally disposed waste \nand costs of illegal disposal of waste",
             x="Waste volume, cubic meters",
             y="Cost per cubic meter, RUB") +
        xlim(c(0, max(new_data$V)))+
        ylim(c(0, Ta*1.5))
        
    
    # add a footnote about paramaters used for the current plot
    profit <- arrangeGrob(profit, 
                          sub = textGrob(note, 
                                         x = 0, 
                                         hjust = -0.1, 
                                         vjust=0.1, 
                                         gp = gpar(fontface = "italic", fontsize = 12)))
    
    # show plot
    print(profit)

}


simDumping <- function(max_waste_volume = 2000, 
                       Illegal_dumping_fine_P = 300000,
                       Illigal_landfilling_fine_P1 = 500000,
                       Fine_probability_k = 0.5,
                       Official_tax_Ta = 600) {
    
    ### this function creates interactive plot
    
    max_waste_volume <<- max_waste_volume 
    Illegal_dumping_fine_P <<- Illegal_dumping_fine_P
    Illigal_landfilling_fine_P1 <<- Illigal_landfilling_fine_P1
    Fine_probability_k <<- Fine_probability_k
    Official_tax_Ta <<- Official_tax_Ta
    
    manipulate(suppressWarnings(plotDumping(max_waste_volume, 
                           Illegal_dumping_fine_P,
                           Illigal_landfilling_fine_P1,
                           Fining_probability_k,
                           Official_tax_Ta)
                           ),
               
               # set up sliders
               max_waste_volume = slider(0, 50000, 
                                         initial = max_waste_volume, 
                                         step = 100,
                                         label = 'X axis range'),
               Illegal_dumping_fine_P = slider(0, 5000000, 
                                               initial = Illegal_dumping_fine_P,
                                               step = 10000, 
                                               label = 'Illegal dumping fine (P)'),
               Illigal_landfilling_fine_P1 = slider(0, 5000000, 
                                                    initial = Illigal_landfilling_fine_P1, 
                                                    step = 10000,
                                                    label = 'Illegal landfilling fine (P1)'),
               Fining_probability_k = slider(0, 1, 
                                             initial = 0.5,
                                             step = 0.01,
                                             label = 'Probability of being fined (k)'),
               Official_tax_Ta = slider(0, 3000, 
                                        initial = Official_tax_Ta, 
                                        step = 50,
                                        label = 'Official waste disposal tax (T)')
    )
}

simDumping() # for reasons unknown I have to run this function twice to get proper interactive plot

Signs of Scientific Respect

Today I received a copy of proceedings of the conference I participated in. A peculiar moment is that my article about using Random Forest algorithm for the illegal dumping sites forecast is the very first article of my section (as well as of the whole book) and it was placed regardless of the alphabetical order of the family names of the authors (this order is correct for all other authors in all sections).

My presentation and speech were remarkable indeed - the director of my scientific-research centre later called it "the speech of guru" (actually, not a "guru", there is just no suitable equivalent in English for the word used). Also the extended version of this article for one of the journals of the Russian Academy of Sciences received an extremely positive feedback from the reviewers. So I suppose the position of my article is truly some kind of respect for the research and presentation and not a random editorial mistake.

Now I should overcome procrastination and make a post (or most likely two) about this research of mine.

Sverdlovskaya Region, Russia: a Field Survey in Autumn. Part 2: Scars on the Earth

Ok, it's time to finish the story about land monitoring in Sverdlovskaya region. In this post I would like to demonstrate some of the most unpleasant types of the land use.

Lets begin with illegal dumping. This dump (note that there is the smoke from waste burning down) is located right next to the potato field (mmm... seems these  potato are tasty). The ground was intentionally excavated here for dumping waste. Obviously this dump is exploited by the agricultural firm - owner of this land, but who cares...

Panorama of freshly burnt illegal dump

The next stop is peat cutting. A huge biotops are destroyed for no good reason (I can't agree that use of peat as an energy source is a good one). At the picture below you can see a peat cutting with the area of 1402 ha. There are dozen of them in the study area...

Peat Cutting (RapidEye, natural colours)

But the most ugly scars on the Earth surface are left from mining works. There is Asbestos town in Sverdlovskaya region. It was names after asbestos that is mined  there. The quarry has an area of 1470 ha and its depth is over 400 meters. Its slag-heaps covers another 2500 ha... The irony is that this quarry gives a job for this town and killing it. You see, if you wand to dig dipper you have to make quarry wider accordingly. Current depth is 450 m and in projects it is over 900 m, but the quarry is already next to the living buildings. So quarry is going to consume the town... By the way, the local cemetery was already consumed. Guess what happened to human remains? Well, it is Russia, so they were dumped into the nearest slug-heap.

Here is the panorama of the quarry. You may try to locate BelAZ trucks down there ;-)

Asbestos quarry

Here is the part of the biggest slag-heap:

A slag-heap

That's how it looks from space:

Asbestos town area (imagery - RapidEye, NIR-G-B pseudo-colour composition)

And in the end I will show you the very basic schema of disturbed land in the study area (no settlements or roads included). Terrifying isn't it? 

Basic schema of disturbed land 

Conference Announcement: Open GIS!

Open GIS! is a conference for users and developers of open-source GIS. It will take place on November 17-18, 2012 in Moscow, Russia. The really cool thing is that participation in this crowd-source conference is free of charge!!! Of course modest donations will be appreciated and a sponsorship as well, but it is not mandatory - we want this event to be available for everyone who are interested in open geospatial software and data. Download leaflet!

A call for papers will be opened in August but already now you may propose topics of your talk or master-class to the committee.

Please, spread the word!

P.S. If you are (a rear) person who interested in illegal dumping research (from geospatial point of view) - come to this conference to hear my talk on it!

Spatial Randomness Evaluation in R: Monte Carlo Test

This post is a some kind of reply to this one.

So our goal is to determine whether our point process is random or not. We will use R and spatstat package in particular. Spatstat provides a very handy function for this, that uses K-function combined with Monte Carlo tests. I will spear you from burbling  about theory behind it - the necessary links were already provided. Lets get directly to action.

In this example I will test data about location of my "favourite" illegal dumps in St. Petersburg and Leningrad region.

# we will need: 

library(maptools) 

library(rgdal) 

library(spatstat)

# import data for analysis

S <- readShapePoints("custom_path/dump_centroids.shp", proj4string= CRS("+proj=tmerc +lat_0=0 +lon_0=33 +k=1 +x_0=6500000 +y_0=0 +ellps=krass +towgs84=23.92,-141.27,-80.9,-0,0.35,0.82,-0.12 +units=m +no_defs"))

SP <- as(S, "SpatialPoints")

P <- as(SP, "ppp")

# perform the test itself with a 100 simulations

E <- envelope(P, Kest, nsim = 100)

plot(E, main = NULL)

And here is what we've got in the end, a fancy graph, which demonstrates that our data (Kobs) significantly deviates from a random process (Ktheo):

Illegal Dumping in Leningrad Region: Press Conference

There was a press conference on Tuesday the 19-th about illegal dumping in Leningrad region (Russia). I was asked to be the main speaker there and to present to the press my recent study on illegal dumping prevention. I've already had two presentations on this subject recently at the international scientific conference in St. Petersburg State University and at the round table for the discussion of the upcoming "Let's do it. Russia" clean up event.

Some video from the press conference:

The main conclusion that I made by investigating possible impacts on illegal dumping prevention (such as penalty increase, chance of being caught increase and waste disposal fare decrease) is that decrease of the waste disposal fare for population is the most efficient way. And I managed to find two other publications that came to the exact conclusion (for example, there is an evidence that 1% waste fare increase leads to 3% increase of illegal dumping cases).

By the way I was able to assess probability of being caught for illegal dumping in Russia. It is about 10^-5 (you can die while playing soccer with such probability).

The only way to reduce waste fares is to use waste as a resource. That means that the only way to prevent illegal dumping is to create waste management system that would be able to complete the zero waste goal.

And here is an abstract from my article:

Mechanisms of the land protection were discussed in this article. An algorithm of decision making whether to dump illegally or not was explained. Formulas for determination of profitable ration of expenditures per unit and amount of illegally dumping waste are substantiated. Effect from different types of impacts that can be used for land protection from illegal dumping were discussed (such as fares change, penalties change, penalty application probability change). Decreasing of waste disposal fares was acknowledged as the most effective way for illegal dumping prevention, but it is possible only if «zero waste» concept is implemented.

Legal and Illegal Dumps in Voronezhskaya Region, Russia

I couldn't believe my eyes when I saw that map! The true situation with waste management uncovered at the official geoportal of Voronezhskaya region.

The geoportal itself is quite good especially for Russia and thare are a lot of information. It is even possible to download some of the data and almost all metadata. And because of their kindness we can see the ugly face of true Russian environmental ignorance and corruption:

Dumps in Voronezhskaya region: Red circles - illegal dumps; Green circles - officially allowed, unlicensed dumps; Squares with circles inside - landfills

First of all - you may see how many red circles out there. These are my "favourite" illegal dumps... terrible indeed. 

But look at all these green circles and don't let them trick you: green here does not stand for "green". These are the same as illegal dumps, but... legal! Yes, these are dumps and their owners have no license for waste treatment and will never have because the soil and ground waters are not protected there.

So we have 9 landfills there. But not all of them have licence too... And finally there are no waste treatment plant at all.

Here you are an ugly inconvenient truth about dumps in Russia. Thanks to administration for shearing with us, but too bad they paint the same shit in different colours.

Methodology for Assessment of Environmental Risk Caused by Fires at Illegal Dumps

It's actually already two month old news, but my research "Developement of the Universal Methodology for Assessement of Environmental Risk Caused by Fires at Illegal Dumps" (download in RUSSIAN), that was made special for Fire Monitoring Challenge (by GIS-Lab, Microsoft, NEXTGIS, several universities and GIS/spatial data corporations), was  awarded the 2-nd pace. The prize consisted of the fancy diploma, Lenovo IdeaPad G560 (thanks to all the gods it became much less uglier when I've installed openSUSE at it and applied an OSM sticker ;-), a wireless mouse (my wife was happy to grab it) and a nice book on remote sensing for children.

Instead of abstract:
Developed methodology for assessment of the fire probability in dependence of spatial location and actual area of illegal dump. It is applicable for any part of the world. Software used: QGIS, R.

Spatial component of the probability of the fire at illegal dump in Leningrad region, Russia

I was lucky to present this research at two conferences and today I've received a printed "minor" publication of the article (it is beta-version of the paper available at the link above). So it is possible now to cite it as:

Yury V. Ryabov (2011) Razrabotka univercal'noy metodiki rascheta veroyatnosti vozniknovenia pozhara na nesankcionirovennoy svalke // Sbornik nauchnih trudov molodyh specialistov, prepodavateley i aspirantov po resultatam provedenia Tret'ego molodezрnogo ecologichescogo congressa "Severnaya palmira", 21-22 noyabria 2011, Sankt-Peterburg. - SPb NICEB RAN - pp. 93-106.

To Do: develope formula for composition coefficient calculation; translation to English; major publication.

P.S. If you are interested in this research and do not speak Russian don't hesitate to contact me and ask for general translation.

A Study on Illegal Dumpimg Prediction

There is an interesting note about GIS model for illegal dumping occurrence prediction. Researches had an assumption that high accessibility of the site and its low visibility will determine the probability of illegal dumping occurrence. They reported that unfortunately this assumption was wrong and there are no significant influence of these factors on illegal dumping. 

Actually such result is not a surprise for me. Most of the huge dumping sites in St. Petersburg and Leningrad region I'm aware of do not have very high accessibility, I would say that their accessibility is somewhat moderate. Relatively low accessibility affects "visibility" of the dumping sites and I think that these two parameters are correlated. Also it isn't clear if researchers defined the amount of waste that separates illegal dump from just litter. It seems that they haven't used such parameter so their studies could be affected by litter, which can be found almost anywhere.

Illegal Dumping and Cadastre

As a specialist in land monitoring and cadastre I always suspected that uncertainty in legal status of the land parcel may provoke illegal dumping at such parcel. One evidence I collected during the study on implementation of high resolution imagery for monitoring of illegal dumping - illegal landfill occurred on the land parcel with no particular legal status. Today I found another one - illegal dump existed for several years because the land parcel owner wasn't determined.

WorldView-2 Imagery for Illegal Dumping Monitoring

Digital Globe published all research papers that was submitted to 8-Brand Challenge. And you can find mine there ;-)

Yes, this research on Illegal Dumping Monitoring With Implementation of WorldView-2 Imagery isn't brilliant (my skills in remote sensing and English could be better), but if you are interested in illegal dumping monitoring it may provide you with some insights. And don't hesitate to contact me if you would like to cooperate in illegal dumping researches.

One of the most interesting finding of the research is that it is hard to distinguish illegal landfill from the construction site (which is crucial for St. Petersburg). So it is necessary to use cadastral data to determinate type of the land use of the land parcel (cadastre contains information if there are construction works at the given parcel).

Mean values of digital numbers for Illegal Landfill,Construction Site and Constructions (buildings) at WV-2 imagery.  

Also I wasn't able to test Change Detection method (using Non-Homogeneous Future Difference index calculation method developed for WV-2) properly, because I haven't ordered multi-temporal imagery in the first place... But seems that it can provide some advantages. See the paper for more information.

FIRMS for Detection of Fires at Illegal Dumps

Today I've acquired another evidence that FIRMS data can be used for monitoring of fires at illegal dumps. Of course I was sure it is possible and I was able to detect fires occurred at one of the landfills in St. Petersburg, but I had no evidences that fires occurred next to illegal dumps with known [to me] location were indeed fires at illegal dumps, because I do not have an information about dumps age and FIRMS accuracy about 1 km and exceeds common size of dump, which is about couple of dozens or hundreds of meters, so there is no way I could insist that fire points next to illegal dumps were actually fires at illegal dumps.

But I've found some good (actually - bad) news today. There is a note about fires at two illegal dumps in Leningrad region. Look what we've got:

Despite no points located at the illegal dumps directly they lie within 1 km zone from the borders of dumps and the news show that these fires are actually dump fires. The bad thing is that these fires lasts for a several days, but were detected just one and two times.

3 cool events of the week

There were 3 very interesting events on this week (during my official vacation):

• a Russian - Finnish workshop dedicated to our project related to management of hazardous part of household waste.
• An "International" conference named "An Environmental Equilibrium"; and...
• OSM mapping party in Volhov town.

A workshop passed as always: Russians posed themselves as know-all persons (despite no one was prepared because all the materials were in English...), talked a lot but ineffective; and Finnish did the real job. For the first time in my life I had to be a parallel Russian-Finnish and a Finnish-Russian translator. A hard job, I have to admit...

"An Environmental Equilibrium" conference (which first day I missed because of the workshop) exceeded my expectations... a little, actually, but nevertheless. The funniest thing was that the first face I saw when entered the lecture hall was the face of my scientific adviser, who new that I will participate but didn't tell me that he will be a participant too O_o. He leaved quickly so we were unable to talk.

There were not as many people as expected (because of the summer time and a poor feedback from organisers, who didn't provided participants with the time table in advance, I suppose the main part of scientists who sanded papers preferred to be somewhere else) - organisers even had to merge together all the sections because of the lack of speakers. The good thing was that the publications were ready and I was able to enjoy my article on the subject of illegal dumping monitoring at St. Petersburg with implementation of high-resolution satellite imagery. The bad thing was that my scientific adviser had already left the conference when I demonstrated my presentation.

An unexpected finding was that the map in my Garmin, which expected to be fresh, had a poor an outdated information about location of the conference and OSM was also poor, but up to date.

OSM mapping party in Volhov was my first mapping party. Despite there were few  participants (actually if there were more of them I wouldn't be able to participate at all because in that case party would took place a week earlier, when I was on vacation in Narva... and mapped it; collected data became more important due to imagery for Narva won't be available soon) and heavy rain (from time to time) we were able to survey western part of town.

After 2 hours of mapping we gathered at the cafe to discuss our survey. I've made an attempt to find an answer for the the question about mapping managed green areas, but it seems that I'm the only one who care.

The most unpleasant part was an attention from drunken ex-prisoner in cafe, who started to ask questions about our activities and then begged for money for beer. In Narva my wife and I brought attention of the local drunkards while survey too...

In the end we had a nice attraction on board of Zverik's jeep - a survey of the Russian roads ;-)

musora.bolshe.net launched 500 actions to get rid of illegal dumps

Musora.bolshe.net [No.more.waste - my unofficial translation to English] is another environmental-oriented NGO in Russia. Fellows try to solve multiple waste problems like separate waste collection and illegal dumping (but not illegal landfilling - unfortunately, this fish is not for their teeth for now) in countries of the former USSR.

Guys were inspired by Estonian Let's do it project and its international offspring and decided to carry out 500 actions to get rid of illegal dumps on the 15 of May.  Hope that information about locations and composition of illegal dumps they're asking to provide is going to be reflected at the Global waste map. 

Musora.bolshe.net created a map for the scheduled actions. There are only 3 actions at the moment, hope this number will grow. You can pick up an action from this map and take part in it.

P.S. When I discovered Global waste map today there were only a couple of places with litter in St. Petersburg and whole Russia. Immediately I decided to reveal some of the "inconvenient truth" to the civilised world and upload data about large illegal landfill inside the borders of St.Petersburg  and a thematic "photo" (RGB true-color + panchrom composition from WorldVeiw-2 imagery). Seems, it is the largest illegal landfill in Europe at this map. Unfortunately I have to register (don't know if I will) before I will be able to edit composition of the illegal landfill.

40 000 m³ illegal landfill inside St. Petersburg