This is the first of three blog posts about working with data sets in Maple.

In 2013, I wrote a library for Maple that used the HTTP package to access the Quandl data API and import data sets into Maple. I was motivated by the fact that, when I was downloading data, I often used multiple data sources, manually updated data when updates were available, and cleaned or manipulated the data into a standardized form (which left me spending too much time on the data acquisition step).

Simply put, I needed a source for data that would provide me with a searchable, stable data API, which would also return data in a form that did not require too much post-processing.

My initial library had really just scratched the surface of what was possible.

Maple 2015 introduced the new DataSets package, which fully integrated a data set search into core library routines and made its functionality more discoverable through availability in Maple’s search bar.

Accessing online data suddenly became much easier. From within Maple, I could now search through over 12 million time series data sets provided by Quandl, and then automatically import the data into a format that I could readily work with.

If you’re not already aware of this online service, Quandl is an online data aggregator that delivers a wide variety of high quality financial and economic data. This includes the latest data on stocks and commodities, exchange rates, and macroeconomic indicators such as population, inflation, unemployment, and so on. Quandl collects both open and proprietary data sets from many sources, such as the US Federal Reserve System, OECD, Eurostat, The World Bank, and Open Data for Africa. Best of all, Quandl's powerful API is free to use.

One of the first examples for the DataSets package that I constructed was in part based on the inspirational work of Hans Rosling. I was drawn in by his ability to use statistical visualizations to break down complex multidimensional data sets and provide insight into underlying patterns; a key example investigating the correlation between rising incomes and life expectancy.

As well as online data, the DataSets package had a database for country data. Hence it seemed fitting to add an example that explored macroeconomic indicators for several countries. Accordingly, I set out to create an example that visualized variables such as Gross Domestic Product, Life Expectancy, and Population for a collection of countries.

I’ll now describe how I constructed this application.

The three key variables are Gross Domestic Product at Power Purchasing Parity, Life Expectancy, and Population. Having browsed through Quandl’s website for available data sets, the World Bank and Open Data for Africa projects seemingly had the most available relevant data; therefore I chose these as my data sources.

Pulling data for a single country from one of these sources was pretty straight forward. For example, the DataSets Reference for the Open Data for Africa data set on GDP at PPP for Canada is:

DataSets:-Reference("quandl", "ODA/CAN_PPPPC"));

In this command, the second argument is the Quandl data set code. If you are on Quandl’s website, this is listed near the top of the data set page as well as in the last few characters of the web address itself: https://www.quandl.com/data/ODA/CAN_PPPPC . Deconstructing the code, “ODA” stands for Open Data for Africa and the rest of the string is constructed from the three letter country code for Canada, “CAN”, and the code for the GDP and PPP. Looking at a small sample of other data set codes, I theorized that both of the data sources used a standardized data set name that included the ISO-3166 3-letter country code for available data sets. Based on this theory, I created a simple script to query for available data and discovered that there was data available for many countries using this standardized code. However, not every country had available data, so I needed to filter my list somewhat in order to pick only those countries for which information was available.

The script that I had constructed required three letter country codes. In order to test all available countries, I created a table to house the country names and three-letter country codes using data from the built-in database for countries:

ccdata := DataSets:-Builtin:-Reference("country")[.., "3 Letter Country Code"];

cctable := table([seq(op(GetElementNames(ccdata[i])) = ccdata[i, "3 Letter Country Code"], 
                                i = 1 .. CountRows(ccdata))]):

My script filtered this table, returning a subset of the original table, something like:

Countries := table( [“Canada” = “CAN”, “Sweden” = “SWE”, … ] );

You can see the filtered country list in the code edit region of the application below.

With this shorter list of countries, I was now ready to download some data. I created three vectors to hold the data sets by mapping in the DataSets Reference onto the “standardized” data set names that I pulled from Quandl. Here’s the first vector for the data on GDP at PPP.

V1 := Vector( [ (x) -> Reference("quandl", cat("ODA/", x, "_PPPPC"))

                   ~([entries(Countries, nolist, indexorder)])]):

#Open Data for Africa GDP at PPP

Having created three data vectors consisting of 180 x 3 = 540 data sets, I was finally ready to visualize the large set of data that I had amassed.

In Maple’s Statistics package, BubblePlots can use the horizontal axis, vertical axis and the relative bubble size to illustrate multidimensional information. Moreover, if incoming data is stored as a TimeSeries object, BubblePlots can generate animations over a common period of time.

Putting all of this together generated the following animation for 180 available countries.

This example will be included with the next version of Maple, but for now, you can download a copy here:DataSetsBubblePlot.mw

*Note: if you try this application at home, it will download 540 data sets. This operation plus the additional BubblePlot construction can take some time, so if you just want to see the finished product, you can simply interact with the animation in the Maple worksheet using the animation toolbar.

A more advanced example that uses multiple threads for data download can be seen at the bottom of the following page: https://www.maplesoft.com/products/maple/new_features/maple19/datasets_maple2015.pdf You can also interact with this example in Maple by searching for: ?updates,Maple2015,DataSets

In my next post, I’ll discuss how I used programmatic content generation to construct an interactive application for data retrieval.

The Joint Mathematics Meetings are taking place this week (January 6 – 9) in Seattle, Washington, U.S.A. This will be the 99th annual winter meeting of the Mathematical Association of America (MAA) and the 122nd annual meeting of the American Mathematical Society (AMS).

Maplesoft will be exhibiting at booth #203 as well as in the networking area. Please stop by our booth or the networking area to chat with me and other members of the Maplesoft team, as well as to pick up some free Maplesoft swag or win some prizes.

Given the size of the Joint Math Meetings, it can be challenging to pick which events to attend. Hopefully we can help by suggesting a few Maple-related talks and events:

Maplesoft is hosting a catered reception and presentation ‘Challenges of Modern Education: Bringing Math Instruction Online’ on Thursday, January 7th at 18:00 in the Cedar Room at the Seattle Sheraton. You can find more details and registration information here: www.maplesoft.com/jmm

Another not to miss Maple event is “30 Years of Digitizing Mathematical Knowledge with Maple”, presented by Edgardo Cheb-Terrab, on Thursday, January 7 at 10:00 in Room 603 of the Convention Center.

Here’s a list of Maple-related events and talks:

Exploration of Mathematics Teaching and Assessment through Maple-Software Projects of Art Diagram Design as Undergraduate Student Research Projects

Wednesday, Jan 6, 10:20, Room 2B, Convention Center

Lina Wu

30 Years of Digitizing Mathematical Knowledge with Maple

Thursday, Jan 7, 10:00, Room 603, Convention Center

Edgardo Cheb-Terrab

MAA Poster Session – Collaborative Research: Maplets for Calculus

Thursday, Jan 7, 14:00, Hall 4F, 4th Floor, Convention Center

Challenges of Modern Education: Bringing Math Instruction Online

Thursday, Jan 7, 18:00, Cedar Room, 2^nd Floor, Sheraton Center

Using Maple to Promote Modelling in Differential Equations

Friday, Jan 8, 10:40, Room 617, Convention Center

Patrice G Tiffany; Rosemary C Farley

If you are presenting at Joint Math and would like to advertise your Maple-related talk, please feel free to comment below, or send me a message with your event and I’ll add it to the list above.

See you in Seattle!

Daniel

Maple Product Manager

As the year draws to a close, we start looking forward to a new year and a new release of Maple. With every new release comes many new features and updates to explore.

We are looking for several new beta testers with a good working knowledge of Maple; We need your input, your ideas, and your experience with our products to help us improve the software and get it ready for general release.

There are many benefits to becoming a beta tester:

• You’ll get to use the new software before anyone else does.
• You’ll help us make our software better in ways that work for you.
• Your suggestions could determine the future direction of the software.
• You’ll get feedback right from the development team.

If you are interested in becoming a beta tester for the next version of Maple, please email: beta (at) maplesoft.com for more information.

We are looking for enthusiastic Maple users to become Maple Ambassadors, to inspire and educate others about the benefits that Maple brings to education.

As an Ambassador, you will have the opportunity to influence the development of Maple through regular meetings with Maplesoft developers, get advance news of upcoming features and products, get assistance with Maple events on your campus, and more. In return, we ask that you do what you are probably already doing – sharing your experiences with Maple, answer questions on forums (like this one!), sharing your Maple applications, providing us with feedback, etc.

You can find more information and an application form at Maple Ambassador Program. We’re looking forward to hearing from you!

Daniel

Maple Product Management

Maple’s Code Generation makes it possible to translate your Maple code to various other programming languages including C, Python, and several others. In Maple 2015, we added a new Code Generation target to one of my other personal favourite languages, R. R is a programming language designed for statistical computing and graphics, so no code translation from Maple to R would be complete without attempting to translate as many commands as possible from Maple’s Statistics package. 

Translating code from one language to another is tricky business. Maple 2015 represented the first time that any Code Generation target language added the ability to translate commands from the Statistics package. With R, we found that many common statistics commands had almost a one-to-one mapping, such as Statistics:-Mean = mean, but several others were much more complicated, including several commands for dealing with probability functions that did not have direct mappings due to differences in how the systems handle symbolic probability functions.

A list of statistics commands that can be translated from Maple to R can be found here.

In addition to assisting me recall the correct syntax in R, having worked with CodeGeneration[R] for several months now, I find that one of my most common uses for Maple’s code generation to R is simply to pass data between the systems. A simple example:

 CodeGeneration:-R( LinearAlgebra:-RandomMatrix( 5, 2 ) );

translates to the following in R:

 cg <- matrix(c(-4,27,8,69,99,29,44,92,-31,67),nrow=5,ncol=2)

To see a couple more short examples, here’s a short video that I recorded on Code Generation to R:

A little known fact about Code Generation is that the translation files can be viewed in from the “samples” directory in your Maple install directory. Similar to many of Maple’s packages, you can view all of the source code that Code Generation uses for its translations. For example, you can view the translations for the commands that I mentioned above from the “FunctionTable.mm” file inside of your “%MapleInstallDir%/ samples/CodeGeneration/R” directory.

Should you have any feedback on this translation, or any other, please feel free to contact us. We’re also on the hunt for our next code generation targets, so let us know what other languages you would like to see added as Code Generation targets.