A GoT Introduction to the tidyverse

In this

In this exercice we will practice a bit of R, exploring data from the Game of thrones series with the dplyr and sf libraries and we will visualize them using the ggplot2 library. The tabular data used was collected by Jeffrey Lancaster and can be found this project. The spatial data was created by ESRI and is available [here](https://www.arcgis.com/home/item.html?id=43d03779288048bfb5d3c46e4bc4ccb0?]. To start you can clone the repository https://github.com/comeetie/got to retrieve all the necessary data.

Exercise 1 : load some data and look at their structures

Data import

Import characters.csv, episodes.csv, scenes.csv and appearances.csv files from the data directory and store them in variables of the same name.

Indices

Use read_csv() preferably from readr library for a better data type recognition.

Solution

library(readr)
characters = read_csv("data/characters.csv")
episodes = read_csv("data/episodes.csv")
scenes = read_csv("data/scenes.csv")
appearances = read_csv("data/appearances.csv")

data structure

Look at the size of the data.frame appearances, observe the variables common to the scenes and appearances tables. Use the str and summary functions on the different tables to understand their structures and relationships.

Indices

Use nrow, ncol or dim to know the dimensions of the data.frame. Use the names function to know the names of the columns and the %in% operator or the intersect function for example to find common variables.

Solution

dim(appearances)

[1] 12114     2

appearances_cols= names(appearances) 
appearances_cols[appearances_cols %in% names(scenes)]

[1] "sceneId"

summary(characters)

     name               sex               house             killedBy        
 Length:587         Length:587         Length:587         Length:587        
 Class :character   Class :character   Class :character   Class :character  
 Mode  :character   Mode  :character   Mode  :character   Mode  :character  
    image          
 Length:587        
 Class :character  
 Mode  :character  

str(characters)

tibble [587 × 5] (S3: spec_tbl_df/tbl_df/tbl/data.frame)
 $ name    : chr [1:587] "Addam Marbrand" "Adrack Humble" "Aeron Greyjoy" "Aerys Targaryen" ...
 $ sex     : chr [1:587] "male" "male" "male" "male" ...
 $ house   : chr [1:587] NA NA "Greyjoy" NA ...
 $ killedBy: chr [1:587] NA NA NA NA ...
 $ image   : chr [1:587] NA NA "https://images-na.ssl-images-amazon.com/images/M/MV5BNzI5MDg0ZDAtN2Y2ZC00MzU1LTgyYjQtNTBjYjEzODczZDVhXkEyXkFqcG"| __truncated__ NA ...
 - attr(*, "spec")=
  .. cols(
  ..   name = col_character(),
  ..   sex = col_character(),
  ..   house = col_character(),
  ..   killedBy = col_character(),
  ..   image = col_character()
  .. )

Exercise 2 : some R base commands

Sum

Use the scenes table to calculate the number of dead characters in the whole series.

Indices

Use the $ operator and the sum function.

Solution

sum(scenes$nbdeath)

[1] 376

Conditional selection

Use the scenes table to calculate the number of dead characters in the first season.

Indices

Use the $ operator and the sum function as above, but select the correct rows using a boolean vector constructed using the id episode column.

Solution

sum(scenes$nbdeath[scenes$episodeId<=10])

[1] 32

Counting

Use the characters table to find the 5 biggest murderers of the series.

Indices

Use the table and sort functions

Solution

sort(table(characters$killedBy),decreasing = TRUE)[1:5]

          Jon Snow Daenerys Targaryen         Arya Stark     Sandor Clegane 
                12                 11                 10                  9 
  Cersei Lannister 
                 7 

Selection

Find the length of the longest scene and the id of the episode.

Indices

Use which.max.

Solution

scenes[which.max(scenes$duration),]

# A tibble: 1 x 9
  sceneStart sceneEnd location    subLocation     episodeId duration   nbc sceneId nbdeath
  <time>     <time>   <chr>       <chr>               <dbl>    <dbl> <dbl>   <dbl>   <dbl>
1 45'58"     56'59"   The Crownl… Outside King's…        73      661    18    3795       0

Exercise 3 : dplyr

Arrange

Find the longest scene duration and episode id using dplyr this time.

Indices

Use arrange,desc and head.

Solution

library(dplyr)
scenes %>% arrange(desc(duration)) %>% head(1)

# A tibble: 1 x 9
  sceneStart sceneEnd location    subLocation     episodeId duration   nbc sceneId nbdeath
  <time>     <time>   <chr>       <chr>               <dbl>    <dbl> <dbl>   <dbl>   <dbl>
1 45'58"     56'59"   The Crownl… Outside King's…        73      661    18    3795       0

Join

Find the characters in the longest scene.

Indices

Use the left_join function to join the scenes and appearences tables.

Solution

scenes %>% arrange(desc(duration)) %>% head(1) %>% left_join(appearances)

# A tibble: 18 x 10
   sceneStart sceneEnd location subLocation episodeId duration   nbc sceneId nbdeath name 
   <time>     <time>   <chr>    <chr>           <dbl>    <dbl> <dbl>   <dbl>   <dbl> <chr>
 1 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Tyri…
 2 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Grey…
 3 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Samw…
 4 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Edmu…
 5 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Arya…
 6 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Bran…
 7 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Sans…
 8 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Brie…
 9 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Davo…
10 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Gend…
11 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Yara…
12 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Robi…
13 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Yohn…
14 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Dorn…
15 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Lord…
16 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Lord…
17 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Lord…
18 45'58"     56'59"   The Cro… Outside Ki…        73      661    18    3795       0 Lord…

Aggregation

Find the most visited place.

Indices

Use group_by and summarize with the function n().

Solution

scenes %>% group_by(location) %>% summarise(nbsc = n()) %>% arrange(desc(nbsc))

# A tibble: 26 x 2
   location           nbsc
   <chr>             <int>
 1 The Crownlands     1252
 2 The North           888
 3 North of the Wall   342
 4 The Wall            309
 5 The Riverlands      301
 6 Meereen             168
 7 Braavos             103
 8 The Vale             54
 9 Dorne                49
10 The Reach            40
# … with 16 more rows

Filer

How many scenes take place in King’s Landing ?

Indices

Use the filter function to target the relevant rows.

Solution

scenes %>% filter(location=="King's Landing") %>% group_by(location) %>% summarise(nbsc = n())

# A tibble: 0 x 2
# … with 2 variables: location <chr>, nbsc <int>

Aggregation

Find the precise location (subLocation) where the most people die?

Indices

Use the sum function during the aggragation and the subLocation feature.

Solution

scenes %>% group_by(subLocation) %>% summarise(nbd=sum(nbdeath)) %>% arrange(desc(nbd))

# A tibble: 97 x 2
   subLocation          nbd
   <chr>              <dbl>
 1 King's Landing        74
 2 Winterfell            60
 3 <NA>                  51
 4 Castle Black          41
 5 The Twins             12
 6 The Haunted Forest     9
 7 The Wall               9
 8 Craster's Keep         8
 9 The Wolfswood          8
10 Dragonstone            6
# … with 87 more rows

Join, filter, aggregate

Find the episode where Jon Snow has the longuest screen time.

Indices

Use the sumfunction during the aggregation and join with the scenes and episodes table to aggregate at the episode level.

Solution

appearances %>%filter(name=="Jon Snow") %>% 
  left_join(scenes) %>% left_join(episodes) %>% 
  group_by(name,episodeId,episodeTitle) %>% 
  summarise(screenTime=sum(duration)) %>% 
  arrange(desc(screenTime)) %>% head(1)

# A tibble: 1 x 4
# Groups:   name, episodeId [1]
  name     episodeId episodeTitle    screenTime
  <chr>        <dbl> <chr>                <dbl>
1 Jon Snow        73 The Iron Throne       2526

Filtrage, numérique

how many characters do have more than 30 minutes of screen time ?

Solution

appearances %>% left_join(scenes)  %>% 
  group_by(name) %>% 
  summarise(screenTime=sum(duration)) %>% 
  filter(screenTime>30*60) %>% 
  nrow()

[1] 98

# en version racourci avec count 
appearances %>% left_join(scenes)  %>% 
  count(name,wt=duration,name = "duration") %>% 
  filter(duration >30*60) %>% 
  nrow()

[1] 98

Join ?

Which characters do have the more scenes together.

Indices

Do a join between the appperences table and the appearences table on the sceneId column then aggregate.

Solution

appearances %>% left_join(appearances,by=c("sceneId"="sceneId")) %>% 
  filter(name.x!=name.y) %>% 
  group_by(name.x,name.y) %>% 
  summarise(nbs=n()) %>% 
  arrange(desc(nbs))

# A tibble: 8,472 x 3
# Groups:   name.x [576]
   name.x             name.y               nbs
   <chr>              <chr>              <int>
 1 Daenerys Targaryen Drogon               172
 2 Drogon             Daenerys Targaryen   172
 3 Daenerys Targaryen Jorah Mormont        148
 4 Jorah Mormont      Daenerys Targaryen   148
 5 Jon Snow           Tormund Giantsbane   145
 6 Tormund Giantsbane Jon Snow             145
 7 Lord Varys         Tyrion Lannister     141
 8 Tyrion Lannister   Lord Varys           141
 9 Davos Seaworth     Jon Snow             136
10 Jon Snow           Davos Seaworth       136
# … with 8,462 more rows

Join ?

Which two characters spend the most time together?

Indices

Make a join of the appearances table with itself on the sceneId column, then a join with the data.frame scenes and an aggregation.

Solution

appearances %>% left_join(appearances,by=c("sceneId"="sceneId")) %>% 
  filter(name.x!=name.y) %>% 
  left_join(scenes %>% select(sceneId,duration)) %>%
  group_by(name.x,name.y) %>% 
  summarise(commonTime=sum(duration)) %>% 
  arrange(desc(commonTime))

# A tibble: 8,472 x 3
# Groups:   name.x [576]
   name.x             name.y             commonTime
   <chr>              <chr>                   <dbl>
 1 Daenerys Targaryen Jorah Mormont           12923
 2 Jorah Mormont      Daenerys Targaryen      12923
 3 Lord Varys         Tyrion Lannister        10764
 4 Tyrion Lannister   Lord Varys              10764
 5 Davos Seaworth     Jon Snow                10380
 6 Jon Snow           Davos Seaworth          10380
 7 Daenerys Targaryen Missandei                9924
 8 Missandei          Daenerys Targaryen       9924
 9 Jon Snow           Tormund Giantsbane       9352
10 Tormund Giantsbane Jon Snow                 9352
# … with 8,462 more rows

Wide format

Build a data.frame with one line per character containing a name column and a column for each place with the duration of presence of each character. If a character has never been in a place the value is equal to 0.

Indices

Calculate the duration of presence by character and location with a group_by and a summary and then use the pivot_wider function of the tidyr library to transform the result into a wide format and fill in the missing values with zeros.

Solution

library(tidyr)
duration_location_character = scenes %>% left_join(appearances) %>% 
  group_by(name,location) %>% 
  summarize(duration=sum(duration))

duration_large = duration_location_character %>% 
  pivot_wider(values_from = duration,names_from = location,values_fill = c("duration"=0))

Matrice

Construct from the previous data.frame a matrix containing only the numerical variables. Filter it to keep only the lines whose sum is higher than 3600. Normalize it so that the sums in lines are equal to 1. Give the name of each character kept to the corresponding line in the matrix with the function rownames.

Indices

Use as.matrix and rowSums

Solution

X=as.matrix(duration_large[,-1])
Xs=X[rowSums(X)>60*60,]
Xns=Xs/rowSums(Xs)
rownames(Xns)=duration_large$name[rowSums(X)>60*60]

Hierarchical clustering

Using the function `dist calculate the manhatan distance between each line of the previous matrix. Then perform a hierarchical clustering with this distance matrix and display the result. You should get a figure similar to the following one :

Solution

hc=hclust(dist(Xns,method="manhattan"))
plot(hc,main = "Clustering of the main characters (geographical profiles)",sub ="@comeetie, 2020",xlab = "")

Exercise 4 : ggplot

A first graphic

Create a jstime table containing for each episode Jon Snow’s screen time and then reproduce this graph :

Indices

Use geom_line

Solution

library(ggplot2)
jstime = appearances %>% filter(name=="Jon Snow") %>% 
  left_join(scenes) %>% 
  group_by(episodeId) %>% 
  summarise(time=sum(duration))
ggplot(jstime) + 
  geom_line(aes(x=episodeId,y=time))+
  theme_bw()+
  xlab("épisode")+ylab("temps")+
  ggtitle("Temps de présence par épisode de John Snow")

Variations

Try other geom’s : area, bars. Compare and comment.

Indices

Use geom_bar but specify that no stats should be claculated with the option stat='identity' and geom_area.

Cumsum

Calculate for the entire scene the cumulative death count and time since the first scene. Then make the following graph:

Indices

Use geom_line and prepare a vector of seasons start time for the labels. Use scale_x_continuous and scale_y_continuous to change the axes labels.

Solution

# nombre de morts cumulé et temps passé
deaths = scenes %>% select(nbdeath,duration,location,episodeId) %>% 
  mutate(t=cumsum(duration),tdeath=cumsum(nbdeath))

# instant de changement de saison
# ? lag
season_t = episodes %>% mutate(ld=lag(total_duration)) %>% 
  mutate(ld=if_else(is.na(ld),0,ld), td = cumsum(ld)) %>% 
  filter(episodeNum==1) %>% pull(td)

# geom_line + labels personalisés
ggplot(deaths) + geom_line(aes(x=t/3600,y=tdeath)) +
  scale_x_continuous("",expand = c(0,0),breaks = season_t/3600,
                     labels =   paste("Saison",1:8),)+
  scale_y_continuous("Nombre de morts cumulés", expand=c(0,0))+
  theme_bw()+
    theme(axis.text.x=element_text(angle=90))+
  ggtitle("Evolution du nombre de mort au cours du temps")

Scatter-plots

Build a data.frame containing for each episode its title, the season, the length of the longest scene, the number of scenes and the number of deaths. Then make a scater plot of the variables number of scenes and longest scene duration.

Indices

Use geom_point

Solution

scenes_stats=scenes %>% left_join(episodes) %>% 
  group_by(episodeTitle,seasonNum) %>% 
  summarize(nb_scenes=n(),duration_max=max(duration),nbdeath=sum(nbdeath))

ggplot(scenes_stats,aes(x=nb_scenes,y=duration_max))+geom_point()

Scatter-plots

Finally, use the color and size of the dots to encode information about the seasons and the number of dead and finalize the graph that might look like this version by setting the scales and adding a few labels.

Indices

USe geom_text and use custom scales and theme.

Solution

labels = scenes_stats %>% filter(duration_max>400|nb_scenes>200)
ggplot(scenes_stats,aes(x=nb_scenes,y=duration_max,col=factor(seasonNum)))+
  geom_point(aes(size=nbdeath))+
  geom_text(data=labels,aes(label=episodeTitle),vjust=-0.6)+
  scale_x_continuous("Nombre de scène",limits = c(0,280))+
  scale_y_continuous("Durée de la scène la plus longue",limits = c(100,800))+
  scale_color_brewer("Saison",palette ="Spectral")+
  guides(colour = "legend", size = "legend")+
  theme_bw()

Box-plots

Make a series of box plots to represent the distributions of scene durations by episode.

Solution

ggplot(scenes %>% left_join(episodes))+geom_boxplot(aes(x=factor(episodeId),y=duration))

Box-plots

Finalize the figure, you can be inspired by the following result:

Solution

labels = scenes %>% filter(duration>400)
ggplot(scenes %>% left_join(episodes))+
  geom_boxplot(aes(x=factor(episodeId),y=duration,fill=seasonNum))+
  geom_text(data=labels ,aes(x=factor(episodeId),y=duration,label=subLocation),hjust = "right",vjust="top")+
  xlab("N° épisode")+
  ylab("Durée des scènes (min)")+
  ggtitle("Répartition des durées des scènes par épisodes")+
  theme_bw()

Stacked bars

Build a table containing for each character and each season the time its screen time. Filter this table to keep only the characters that appear for more than one hour over the seasons. Reorder the levels of the name factor so that the levels are sorted in ascending order of appearance time.

Make a stacked bar-plot of this data identical to the following figure. Use color-brewer to find the palette used.

Solution

screenTimePerSeasons = appearances %>% left_join(scenes) %>% 
  left_join(episodes) %>% 
  group_by(name,seasonNum) %>% 
  summarise(screenTime=sum(duration)) %>% 
  arrange(desc(screenTime)) 
screenTimeTotal = screenTimePerSeasons %>% 
  group_by(name) %>% 
  summarise(screenTimeTotal=sum(screenTime))
mainCharacters = screenTimeTotal %>% 
  filter(screenTimeTotal>60*60) %>% 
  arrange(screenTimeTotal) %>% 
  mutate(nameF=factor(name,levels = name))
data = screenTimePerSeasons %>% left_join(mainCharacters) %>% filter(!is.na(nameF))
ggplot(data)+
  geom_bar(aes(y=nameF,x=screenTime/60,fill=factor(seasonNum,level=8:1)),stat="identity")+
  scale_fill_brewer("Saison",palette = "Spectral")+theme_bw()+
  geom_text(data=mainCharacters,aes(y=nameF,x=screenTimeTotal/60+5,label=paste(round(screenTimeTotal/60),'min')),hjust = "left")+
  scale_x_continuous("Temps d'apparition (min)",breaks = seq(0,750,by=120),limits = c(0,780),expand = c(0,1))+
  ylab("")+ggtitle("Temps d'apparition cumulé par personnage et saison")

Exercise 5 : Spatial data processing with sf

Read spatial data

Read the geographical data describing the GoT universe stored in the data/GoTRelease directory as shapefile files with the sf libraries and load them in data-frames with explicit names, you will specify during the import that the coordinate system is (comparable with) the wgs-84 system whose crs code is 4326. Observe the different tables created: dimensions, variables, type of geometry, metadata.

Indices

Use the st_read function and the `crs agurment and the classic data.frames functions nrow, dim, summary,…

Solution

library(sf)
library(tidyr)
library(ggplot2)
locations=st_read("./data/GoTRelease/Locations.shp",crs=4326)

Reading layer `Locations' from data source `/home/come/Projets/got/data/GoTRelease/Locations.shp' using driver `ESRI Shapefile'
Simple feature collection with 247 features and 5 fields
geometry type:  POINT
dimension:      XY
bbox:           xmin: 5.140883 ymin: -34.58818 xmax: 87.984 ymax: 37.21249
geographic CRS: WGS 84

lakes=st_read("./data/GoTRelease/Lakes.shp",crs=4326)

Reading layer `Lakes' from data source `/home/come/Projets/got/data/GoTRelease/Lakes.shp' using driver `ESRI Shapefile'
Simple feature collection with 19 features and 3 fields
geometry type:  POLYGON
dimension:      XY
bbox:           xmin: 8.288085 ymin: -16.38318 xmax: 85.85195 ymax: 45.10014
geographic CRS: WGS 84

conts=st_read("./data/GoTRelease/Continents.shp",crs=4326)

Reading layer `Continents' from data source `/home/come/Projets/got/data/GoTRelease/Continents.shp' using driver `ESRI Shapefile'
Simple feature collection with 3 features and 2 fields
geometry type:  POLYGON
dimension:      XY
bbox:           xmin: 0.9011003 ymin: -42.00216 xmax: 91.99359 ymax: 49.10222
geographic CRS: WGS 84

land=st_read("./data/GoTRelease/Land.shp",crs=4326)

Reading layer `Land' from data source `/home/come/Projets/got/data/GoTRelease/Land.shp' using driver `ESRI Shapefile'
Simple feature collection with 2 features and 2 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 0.9011003 ymin: -42.00216 xmax: 91.99359 ymax: 49.10222
geographic CRS: WGS 84

wall=st_read("./data/GoTRelease/Wall.shp",crs=4326)

Reading layer `Wall' from data source `/home/come/Projets/got/data/GoTRelease/Wall.shp' using driver `ESRI Shapefile'
Simple feature collection with 1 feature and 2 fields
geometry type:  LINESTRING
dimension:      XY
bbox:           xmin: 16.39357 ymin: 34.79226 xmax: 20.71819 ymax: 35.19238
geographic CRS: WGS 84

islands=st_read("./data/GoTRelease/Islands.shp",crs=4326)

Reading layer `Islands' from data source `/home/come/Projets/got/data/GoTRelease/Islands.shp' using driver `ESRI Shapefile'
Simple feature collection with 86 features and 3 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 5.433233 ymin: -36.04357 xmax: 67.70794 ymax: 37.58262
geographic CRS: WGS 84

kingdoms=st_read("./data/GoTRelease/Political.shp",crs=4326)

Reading layer `Political' from data source `/home/come/Projets/got/data/GoTRelease/Political.shp' using driver `ESRI Shapefile'
Simple feature collection with 12 features and 3 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 0.9011003 ymin: -11.40838 xmax: 26.29015 ymax: 49.10222
geographic CRS: WGS 84

landscapes=st_read("./data/GoTRelease/Landscape.shp",crs=4326)

Reading layer `Landscape' from data source `/home/come/Projets/got/data/GoTRelease/Landscape.shp' using driver `ESRI Shapefile'
Simple feature collection with 37 features and 5 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 5.942558 ymin: -41.92678 xmax: 91.91442 ymax: 47.14478
geographic CRS: WGS 84

roads=st_read("./data/GoTRelease/Roads.shp",crs=4326)

Reading layer `Roads' from data source `/home/come/Projets/got/data/GoTRelease/Roads.shp' using driver `ESRI Shapefile'
Simple feature collection with 21 features and 4 fields
geometry type:  LINESTRING
dimension:      XY
bbox:           xmin: 6.540938 ymin: -24.64828 xmax: 67.18459 ymax: 34.89038
geographic CRS: WGS 84

rivers=st_read("./data/GoTRelease/Rivers.shp",crs=4326)

Reading layer `Rivers' from data source `/home/come/Projets/got/data/GoTRelease/Rivers.shp' using driver `ESRI Shapefile'
Simple feature collection with 74 features and 4 fields
geometry type:  MULTILINESTRING
dimension:      XY
bbox:           xmin: 6.054601 ymin: -36.23881 xmax: 78.73064 ymax: 42.82271
geographic CRS: WGS 84

Distances

Use the st_distance function to calculate distances in m between locations of size5. Which cities are the closest? The farthest ones?

Indices

Don’t forget to filter the location table to keep only the big cities. The function which can be used to find the indices of the maximum and minimum values in a matrix by combining it with the min/max functions. Finally, the functions upper.sort and lower.sort allow to keep only the lower (respectively upper) triangular part of a matrix, this could be useful to find the two nearest cities.

Solution

cities = locations %>% filter(size==5)
dists = st_distance(cities)
cities$name[which(dists==max(dists),arr.ind = TRUE)[1,]]

[1] "Qarth"        "White Harbor"

cities$name[which(dists==min(dists[upper.tri(dists)]) ,arr.ind = TRUE)[1,]]

[1] "Tolos"  "Elyria"

Spatial join

Which family owns the most castles? Use the function st_join to make a spatial join between the location table and the kingdoms table (political).

Solution

CastlesPerKingdoms= st_join(locations,kingdoms) %>% 
  filter(type=="Castle") %>% 
  st_drop_geometry() %>% 
  count(ClaimedBy)
CastlesPerKingdoms

       ClaimedBy  n
1          Arryn 12
2      Baratheon 14
3        Greyjoy  2
4      Lannister 13
5        Martell 17
6  Night's Watch  2
7          Stark 13
8      Targaryen  7
9          Tully 12
10        Tyrell 16
11     Wildlings  1
12          <NA>  5

Binary geometric predicates

Which family owns the most castles? This time if you use the function `st_covers .

Indices

The length of the returned vectors for each polygon is useful. You can apply a function to each element of a list with sapply` andlapply`.

Solution

castles_cover = st_covers(kingdoms,locations %>% filter(type=="Castle"))
kingdoms$nbcastles = sapply(castles_cover,length)
kingdoms %>% arrange(desc(nbcastles))

Simple feature collection with 12 features and 4 fields
geometry type:  MULTIPOLYGON
dimension:      XY
bbox:           xmin: 0.9011003 ymin: -11.40838 xmax: 26.29015 ymax: 49.10222
geographic CRS: WGS 84
First 10 features:
   id             name     ClaimedBy nbcastles                       geometry
1   7            Dorne       Martell        17 MULTIPOLYGON (((10.07742 -9...
2  12        The Reach        Tyrell        16 MULTIPOLYGON (((7.605566 -2...
3   8       Stormlands     Baratheon        14 MULTIPOLYGON (((17.98872 -4...
4   5        The North         Stark        13 MULTIPOLYGON (((10.6842 31....
5  10  The Westerlands     Lannister        13 MULTIPOLYGON (((11.96501 11...
6   2       Riverlands         Tully        12 MULTIPOLYGON (((17.46128 4....
7   9         The Vale         Arryn        12 MULTIPOLYGON (((19.00333 18...
8  11       Crownsland     Targaryen         7 MULTIPOLYGON (((24.62625 7....
9   4      Bran's Gift Night's Watch         2 MULTIPOLYGON (((15.17952 34...
10  6 The Iron Islands       Greyjoy         2 MULTIPOLYGON (((8.194052 13...

Intersections, aggregations

Which family has the largest area of forest?

Indices

The st_intersection function should make your job easier, but don’t forget the st_area function to calculate areas and the basic functions of dplyr.

Solution

st_intersection(kingdoms,landscapes) %>% 
  mutate(area=st_area(geometry)) %>% 
  count(ClaimedBy,wt=area) %>% 
  st_drop_geometry() %>% 
  arrange(desc(n))

       ClaimedBy                  n
1          Stark 1.304682e+12 [m^2]
2      Wildlings 6.000348e+11 [m^2]
3          Arryn 4.988246e+11 [m^2]
4        Martell 4.640324e+11 [m^2]
5      Lannister 3.983993e+11 [m^2]
6      Baratheon 3.724850e+11 [m^2]
7          Tully 1.388066e+11 [m^2]
8         Tyrell 7.842595e+10 [m^2]
9  Night's Watch 4.558375e+10 [m^2]
10     Targaryen 2.750644e+10 [m^2]

The final boss

How far did the main characters of the series (“Jon Snow”, “Tyrion Lannister”, “Daenerys Targaryen”, “Sansa Stark”, “Cersei Lannister”, “Arya Stark”) travel ? The spatial table data/GoTRelease/ScenesLocations.shp will give you the locations of the places referenced in the scene table and you will use the lag function of dplyr.

Indices

Use the lag function after a group_by per character to build a column containing the previous location of the character. Make a double join with the scenes_locations table and use the st_distance function using the by_element option to calculate the distance between two scenes and finally sum it all up with a sum.

Solution

main_char= c("Jon Snow", "Tyrion Lannister","Daenerys Targaryen","Sansa Stark","Cersei Lannister","Arya Stark")
scenes_locations=st_read("./data/GoTRelease/ScenesLocations.shp",crs=4326)

Reading layer `ScenesLocations' from data source `/home/come/Projets/got/data/GoTRelease/ScenesLocations.shp' using driver `ESRI Shapefile'
Simple feature collection with 26 features and 2 fields
geometry type:  POINT
dimension:      XY
bbox:           xmin: 4.143957 ymin: -34.58818 xmax: 87.984 ymax: 38.61921
geographic CRS: WGS 84

distance_characters = scenes %>% left_join(appearances) %>% 
  filter(name %in% main_char) %>%
  group_by(name) %>% 
  mutate(previous_location=lag(location)) %>%
  filter(location!=previous_location) %>%
  left_join(scenes_locations)%>%
  left_join(scenes_locations,by=c("previous_location"="location")) %>% 
  mutate(dist=st_distance(geometry.x,geometry.y,by_element = TRUE)) %>% 
  summarise(total_dist=sum(as.numeric(dist),na.rm=TRUE)/1000)

distance_characters 

# A tibble: 6 x 2
  name               total_dist
  <chr>                   <dbl>
1 Arya Stark             12728.
2 Cersei Lannister        4709.
3 Daenerys Targaryen     61185.
4 Jon Snow               39059.
5 Sansa Stark            11569.
6 Tyrion Lannister       28375.

Another path

Same question but without using the `lag function of dplyr.

Indices

Build a spatial table containing the location of each character’s scenes and their locations in the form of dots. Group by character and use the summary` function to group these points into a multipoint that will keep the order (thedo_union` option should be of interest to you). Convert these points into lines using the st_cast function and calculate the length of these lines.

Solution

distance_characters = scenes %>% left_join(appearances) %>% 
  filter(name %in% main_char) %>% 
  left_join(scenes_locations) %>% 
  st_as_sf() %>%
  group_by(name) %>% summarise(do_union=FALSE) %>% sf::st_cast("LINESTRING") %>% mutate(dist=as.numeric(st_length(geometry))/1000)

distance_characters 

Simple feature collection with 6 features and 2 fields
geometry type:  LINESTRING
dimension:      XY
bbox:           xmin: 4.143957 ymin: -34.58818 xmax: 87.984 ymax: 38.61921
geographic CRS: WGS 84
# A tibble: 6 x 3
  name                                                                     geometry   dist
* <chr>                                                            <LINESTRING [°]>  <dbl>
1 Arya Stark     (14.89991 25.02354, 14.89991 25.02354, 14.89991 25.02354, 14.8999… 12728.
2 Cersei Lannis… (21.22862 5.640294, 14.89991 25.02354, 14.89991 25.02354, 14.8999…  4709.
3 Daenerys Targ… (31.12798 4.632835, 31.12798 4.632835, 31.12798 4.632835, 31.1279… 61185.
4 Jon Snow       (14.89991 25.02354, 14.89991 25.02354, 14.89991 25.02354, 14.8999… 39059.
5 Sansa Stark    (14.89991 25.02354, 14.89991 25.02354, 14.89991 25.02354, 14.8999… 11569.
6 Tyrion Lannis… (14.89991 25.02354, 14.89991 25.02354, 14.89991 25.02354, 14.8999… 28375.

Exercise 6 : cartography with ggplot

Map background

Make a background map of the GoT universe with the lakes, rivers and forests as well as the names of the main cities. You can use geom_sf` andgeom_sf_text` and be inspired by this map:

Indices

Stack the layers, see how to change a ggplot theme and the graph coordinate system options with `coord_sf.

Solution

colforest="#c0d7c2"
colriver="#7ec9dc"
colriver="#87cdde"
colland="ivory"
borderland = "ivory3"  
ggplot()+geom_sf(data=land,fill=colland,col=borderland,size=0.1)+
  geom_sf(data=islands,fill=colland,col="ivory3")+
  geom_sf(data=landscapes %>% filter(type=="forest"),fill=colforest,col=colforest)+
  geom_sf(data=rivers,col=colriver)+
  geom_sf(data=lakes,col=colriver,fill=colriver)+
  geom_sf(data=wall,col="black",size=1)+
  geom_sf_text(data= locations %>% filter(size>4,name!='Tolos'),aes(label=name),size=2.5,family="Palatino", fontface="italic")+
  theme_minimal()+coord_sf(expand = 0,ndiscr = 0)+
  theme(panel.background = element_rect(fill = colriver,color=NA)) +
  labs(title = "GoT",caption = "Etiennne Côme, 2020",x="",y="")

Proportionals symbols

Build a data.frame containing for each location the time of presence on the screen of “Tyrion Lannister”. Load the spatial data data//GoTRelease/ScenesLocations.shp and join it with the previous table. Finally make a map with proportional symbols to visualize these data.

FALSE Reading layer `ScenesLocations' from data source `/home/come/Projets/got/data/GoTRelease/ScenesLocations.shp' using driver `ESRI Shapefile'
FALSE Simple feature collection with 26 features and 2 fields
FALSE geometry type:  POINT
FALSE dimension:      XY
FALSE bbox:           xmin: 4.143957 ymin: -34.58818 xmax: 87.984 ymax: 38.61921
FALSE geographic CRS: WGS 84

Proportionals symbols and facets (1)

Preparation of simplified background maps : Create a spatial data.frame backgound using the function st_as_sf containing a column name with the names of the 6 main characters of the series “Jon Snow”, “Tyrion Lannister”, “Daenerys Targaryen”, “Sansa Stark”, “Cersei Lannister”, “Arya Stark” and an identical geometry column for all characters containing the union of all land and island polygons.

Indices

Utilisez st_geometry pour extraire les géométrie uniquement et st_union plusieurs fois pour combiner les polygones en un unique multi-polygone. Enfin, utilisez st_as_sf pour convertir une data.frame avec une colone geometrie et une colonne name en data.frame spatiale.

Solution

main_char= c("Jon Snow", "Tyrion Lannister","Daenerys Targaryen","Sansa Stark","Cersei Lannister","Arya Stark")
landpol = st_union(st_geometry(land)) 
islandpol = st_union(st_geometry(islands))
backpol=st_union(landpol,islandpol)
background = st_as_sf(data.frame(name=main_char,geometry=rep(backpol,6)))

Proportionals symbols and facets (2)

Create a data.frame with for each main character and each location the time of presence on the screen. Attach this table to the table of georeferenced locations.

Solution

loc_time=appearances %>% filter(name %in% main_char) %>% left_join(scenes) %>% group_by(location,name) %>% summarize(duration=sum(duration,na.rm=TRUE)) 
loc_time_mc = scenes_locations %>% left_join(loc_time)

Symboles proportionels et facettes (2)

Make a series of map (one per character) with their screen times represented in proportional symbols. The final figure might look like this:

Indices

Use the possibilities of facet_wrap with the two previously built data.frame.

Solution

ggplot()+geom_sf(data=background,color=borderland,fill=colland)+
  geom_sf(data=loc_time_mc%>% filter(!is.na(duration)),aes(size=duration/60,color=name))+
  geom_sf_text(data=loc_time_mc%>% filter(duration>60*60),aes(label=location),color="#000000",vjust="bottom",family="Palatino", fontface="italic")+
  coord_sf(expand = 0,ndiscr = 0)+
  scale_color_discrete(guide="none")+
  scale_size_area("Durée (min) :",max_size = 12,breaks=c(30,60,120,240))+
  facet_wrap(~name)+
  theme(panel.background = element_rect(fill = colriver,color=NA),
        text = element_text(family="Palatino",face = "bold",size = 14),
        legend.key = element_rect(fill="#ffffff"),
        ) +
  labs(title = "Temps de présence des personnage principaux",caption = "@comeetie, 2020",x="",y="")

---

---

reproducibility

sessionInfo()

R version 4.0.0 (2020-04-24)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

locale:
 [1] LC_CTYPE=fr_FR.UTF-8       LC_NUMERIC=C               LC_TIME=fr_FR.UTF-8       
 [4] LC_COLLATE=fr_FR.UTF-8     LC_MONETARY=fr_FR.UTF-8    LC_MESSAGES=fr_FR.UTF-8   
 [7] LC_PAPER=fr_FR.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] sf_0.9-6      ggplot2_3.3.2 tidyr_1.1.2   dplyr_1.0.2   readr_1.4.0   knitr_1.30   

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.5         RColorBrewer_1.1-2 pillar_1.4.6       compiler_4.0.0    
 [5] unilur_0.4.0.9000  class_7.3-17       tools_4.0.0        digest_0.6.27     
 [9] evaluate_0.14      lifecycle_0.2.0    tibble_3.0.4       gtable_0.3.0      
[13] pkgconfig_2.0.3    rlang_0.4.8        DBI_1.1.0          cli_2.1.0         
[17] rstudioapi_0.11    yaml_2.2.1         xfun_0.19          e1071_1.7-4       
[21] withr_2.3.0        stringr_1.4.0      generics_0.1.0     vctrs_0.3.4       
[25] hms_0.5.3          classInt_0.4-3     grid_4.0.0         tidyselect_1.1.0  
[29] glue_1.4.2         R6_2.5.0           fansi_0.4.1        rmarkdown_2.1     
[33] farver_2.0.3       purrr_0.3.4        magrittr_1.5       units_0.6-7       
[37] scales_1.1.1       ellipsis_0.3.1     htmltools_0.5.0    assertthat_0.2.1  
[41] colorspace_1.4-1   labeling_0.4.2     KernSmooth_2.23-17 utf8_1.1.4        
[45] stringi_1.5.3      munsell_0.5.0      lwgeom_0.2-3       crayon_1.3.4