Redo Airport Score

-

This morning I have redone everything mentioned in the post 1/8. The last version used an air route dataset of 37,595 records, turned out to be an earlier version. The latest version has 67,240 records.

As a quick reminder, "airport score" is a measurement created based on global airport network. It shows how centered each city is in the global air traffic network. In short, the airport score for each point on earth (or each city) is the sum product of airport weights and inverse distances to the point from all the airports. The airport weights are obtained through eigenvector centrality using a dataset of all the airline linkages in the world.

Method

1. There are 3,300 airports in total, for each city, calculate the distances x to each airport, apply an inverse function of the distance f(x) = 1/(1+x)^p to penalize airports that are further away. Different values of the exponent (p) would produce quite different ranking results. I chose p to be 200 through parameter selection. It produced a ranking that aligns best with GAWC ranking of global cities.

2. Time this f(x) with weights (w) and then sum over the 3,300 airports to get the score for a city. The weights (w) are the relative importance of each airport in the airports' network obtained through eigenvector centrality. Using R package "igraph" can obtain these weights easily. The "importance of each node" is a stationary probability distribution in a network. The intuition behind is that the relative weight of each node is the probability that a traveler who is always traveling in the network among nodes appears in this location.

Improvement

I tried the dataset with 67,240 records once before but the results did not make any sense. Today I realized where the mistake is. That dataset has longitude listed before latitude but I took it for granted that it is latitude - longitude, as in almost everywhere. Ok, this is an excuse.
1. I should not take things for granted
2. Should check the source first

Result

The old version seems incomplete as it focuses more on European cities. The right result from the latest air route dataset has correctly put Atlanta International Airport (ATL) as the top 1 airport. (Hartsfield-Jackson Atlanta International Airport is the world's busiest airport by total movements). The Heathrow airport in London is the second, followed by O'Hare in Chicago and John F. Kennedy in New York. And in terms of city score, New York is on the top of the list. More American cities and Shanghai are in the top 10.

In the map, we can also see higher weights from North America and Asia:

Animation


New ranks of airports and cities in terms of importance in the network.

Compared to the old version:

The city score in Europe has been weakened too (around German and Belgium), compared to the 1/5 post. Animation on the world map:

------
More technical details on how to program the animation in R were discussed in the older post.
Another post discussed how the geological distances were calculated.
Location: New York, NY 10010, USA

Comments

Post a Comment

Popular posts from this blog

How to Draw Heatmap with Colorful Dendrogram

-
Image
Heatmap with Dendrogram: Color Branches and Labels This data visualization example include: * Hierarchical clustering, dendrogram, and heat map based on normalized odds ratios * The dendrogram was built separately to give color to dendrogram’s branches/labels based on clusters using  dendextend * Heatmap is made by  heatmap.2  from  gplots  using the built dendrogram * The rows are sorted by means from highest to lowest, it can be done in either the dendrogram or the  heatmap.2 * Use color palettes from  colorspace Data YRBSS (Youth Risk Behavior Surveillance System) survey data from CDC. This example plots the odds ratios of 63 behavioral questions on one question regarding disordered eating. The odds ratios are median centered by column and log2 transformed Version 1: Color both the branches and labels library (curl) # read file from google drive library (gplots) # heatmap.2 library (dendextend) # make and color dendrogram library (colorspace) #
Read more

Power-law distribution (Pareto)& Zipf's Law: connection and how to fit the distribution of global city population

-
Image
- This post counts for 3 days. - In short,  We use the histogram to outline the distribution, use rank-frequency plot (log-log plot) to identify the power-law distribution, and use maximum likelihood estimation to obtain the parameters. -  I will show 1 st , how the power-law function CDF and PDF can be derived from Zipf CDF; and 2 nd , this least square estimation by excel is wrong and how to get the right one. Historical background on the power-law distribution. - Power-law distribution is as common as normal distribution in the real world. In the 19 th century, Italian economist Pareto realized that 20% of the population owns 80% the wealth, which is the famous 80/20 rule. He later named the power-law distribution as Pareto distribution, which is what we studied in the probability course. - In the 20 th century, another main contributor, Zipf made a similar discovery in linguistics about the frequency of words used. - Besides social wealth, many things follow a hig
Read more

eXtreme Gradient Boosting (XGBoost): Better than random forest or gradient boosting

-
Image
Ensemble Methods 1. Random Forest 2. Gradient boosting 3. Extreme Gradient Boosting Compared to Lasso and Best-subset Lasso Best-subset Compare MSEs Code I recently had the great pleasure to meet with Professor Allan Just and he introduced me to eXtreme Gradient Boosting (XGBoost). XGBoost is “one of the most loved machine learning algorithms at Kaggle”, it somehow combines the advantages of random forest and boosting. We will see later its predictive ability is indeed very strong. In the earlier post  Model Selection using Lasso and Best Subset  I mainly focused on model selection by best-subset, stepwise, or lasso. This post will focus on ensemble method: random forest, Gradient boosting, and eXtreme Gradient Boosting (XGBoost). An ensemble method is a technique that combines the predictions from multiple machine learning algorithms together to make more accurate predictions than any individual model. 1. Random Forest Use library  randomForest . A
Read more