Category: data

Posted on

Why Does Adopting New Information Take So Long?

do we know the question?

As a researcher, I have often heard people lament, “We studied this in the past and nothing was done”, or “Why are we not using this approach”, or some variation concerning the fact that data and information are not being used after the being developed, purchased, or studied.  The question is that we think, using our crystal ball, we have built a masterpiece, and wonder why people don’t adopt our insights.  We often forget that this “knowledge” could be slow to be adopted by others for many reasons.

Failure of adoption:

  • The first is simply the WHY?  Sometimes when doing research we understand more about the question that the person who needs the answer.  So while we prepare our work, we forget our client will only use what they can understand with some level of confidence.  How often have we seen a more senior person misspeak based on information not properly summarized for them?
  • Secondly, there remains the ever consuming “tyranny of the urgent”, in that the research is needed in a timely manner, but the research is not needed beyond the “now”.  The reasons can vary from staff turnover, policy change, new leadership, the findings were not what was expected, to a thousand different reasons.  Furthermore, data is perishable, something that is often forgotten by the researcher, but not the client.
  • Thirdly, the experts may not agree with your opinions.  My wife is a fan of Downton Abbey, and during season 3, Sybil Branson died after childbirth.  The tragedy was there were two doctors arguing over her treatment, and the older doctor stated to the other doctor he is to not interfere.  In many ways, we can find people with good intentions failing to achieve an expected outcome because they are using older models from the past.  They remain uncommitted to learn, and without the application of new information, their working knowledge could, and does, fail, in providing actionable insights, or even providing the wrong information.  Presenting this expert with new information may only lead them to become more entrenched to their position.
  • Finally, our research may not actually answer the question being asked!

I suspect the following challenges will remain with us for a long time, based on  NCHRP Active Implementation: Moving Research into Practice, posted at http://onlinepubs.trb.org/onlinepubs/nchrp/docs/NCHRP_ActiveImplementation.pdf.

For the research community, the ghost of people not adopting our great ideas haunts the adoption of our “great efforts”.  But we must understand what the client may do with the research once it has been delivered, which may depend upon how we communicate before, during and after the research process!

Posted on

Crossing over the U.S-Mexican Border

Recently, Teen Vogue highlighted how crossing the U.S. border remains a daily reality for most people. The author focused on four stories: travel for work, school, shopping and family, although there are other reasons, such as medical or tourism. The article sited a Bureau of Transportation Statistics website which complied numbers, mode and locations crossings into the United States. And BTS does not report illegal crossings…that is Customs and Border Protection’s information.

So in 2018, the largest U.S. crossings with Mexico are shown on the following map

So, there are a few crossings in Southern California, El Paso, and the lower Rio Grande Valley with the largest passenger crossings, mostly by automobile or walking. The largest bus crossings are in Laredo, but the largest Pedestrian and automobile crossings occurred in San Ysidro and El Paso.

No one simply travels across the border just to say, “well I was in “so-and-so””, (well, unless you are interested in joining the Traveler’s Century Club.) Sometimes, we forget that for each data point, there is a reason why someone wanted to enter the United States. But for local and national groups, it is just as important to know the total number and location of how people entered the United States to assist in planning infrastructure and operational needs or to quantify the border’s economic contribution activity.

Notes:

How the graph was created: Downloaded a custom table from BTS Border Crossing Database, converted the file into Excel to fix the geography, and than imported in Tableau.

Related databases:
The BTS information does not include passenger flights, which are reported here

There is some information on commercial freight traffic, such as trains and trucks,which is presented below for 2018. For a fuller comparison of commercial freight transportation across the borders, go TransBorder Freight Data.

Posted on

Does Counting Matter

I am reminded of this Peanuts cartoon. Linus tells Lucy not to count the snowflakes for he already knows the answer. (We can argue that maybe Linus is a seasoned researcher, but I’m sure he was outside earlier doing what Lucy was doing.)


And what is counting, but simply putting a sequence such as “1, 2, 3,…”.  I could type any number on a keyboard and generate data.  When we look at data, sometimes we can get so absorbed in knowing a number that we forget why knowing a number matters.  What benefit is it for Linus or Lucy to know the numbers of snowflakes? No one measures a single snowflake, but rather snowflakes, as the aggregate matters.  For example, one snowflake weighs nothing, but too many can collapse a roof.

There remains a need to count and observe the world, and I am guilty of looking for data when I do research.  So, statistical and data approaches are warranted to make sure we have sound information to make a decision.  While it is easy to measure snowflakes or other actions, sometimes transportation and economic data is not as clearly observed.  Understanding what is needed to be known helps us see the world before we go outside to count snowflakes.

Posted on

International Trade Data, Organizing Data and Client Support

On December 19, I agreed to talk at the New Orleans Data Group. The previous speakers presented work that was largely for internal audiences, so I wanted to focus on something more applied. With my interest in trade, this seemed a good fit.

Globalization refers to interconnected economies, and in a connected world, this largely is tied to investment (such as Foreign Direct Investment), Services, and Goods. (Here is a good place to start if one wants information on U.S. global activity) So, the rest of the talk focused on trade in goods.

There are many reasons why post World War II trade growth blossomed, but the economy benefited from transportation and telecommunication innovations and regulatory changes. (A good discussion of some of this was referred to in Wired magazine’s article “The 20-Ton Packet”)

But trade represents the sum of many things. For example, here are two different industries, such as toys (left) and cut flowers (left). So, there are seasonal patterns to importing merchandise, as there is just as strong seasonal exports of grains and other products. As such, trade represents the sum of decisions between many companies, (a significant amount of trade is intracompany flows).

But we don’t see “TRADE”, we see transportation. Have you ever wondered what was in the truck? The picture on the right was trucks stopped in Louisiana because of a flood in 2016, but most trucks in the Southeast are moving cargo that is intrastate or internal to the Southeast. Trade, as a share of total trade, is actually only 17% of the total traffic activity. However, in Louisiana, as a very strong maritime trade, maritime trade represents a significant industry when compared to other states. (ITTS Working Paper 1 – “International Maritime Trade Benefits the Nation’s Economy“)

There are different pieces of information tied to an international shipment, such as the cargo’s origin and destination, its port of entry, and destination, duties, etc. What is not represented in the chart below is information about the shipper and receiver, the value of the cargo, who is paying for what activity (and is bearing the transportation risks) and other details. The data regarding international trade could thus represent a transaction (generated in respond to an event) or metadata, such as regarding the cargo itself, that is independent of its location and movement. (When conducting trade research, one has to remember U.S. Government disclosure rules do influence how much information can be released.)

Cargo is more likely to move via truck or air for time-sensitive shipments; on the other hand, water tends to handle large, less time-sensitive shipments with lower per unit costs. People make this trade-off. The more expensive the product, the more likely it is that air or trucking will be the
mode selected. Depending on the shipment size, people are more likely to use water, rail or pipeline for similar cargoes. As a result, people’s mode selection is based upon the value, timing, and size of the cargo.

So, there are many things related to trade. The researcher would need to know the details about season patterns, mode, etc., when organizing the data and/or query.

What was delivered…

As part of an ongoing research work, I put several databases into Tableau for the Institute for Trade and Transportation Studies member states. The effort was to develop workbooks with international trade data and other sources of information that may be of use to local planners and economists.
The video about how to engage the dashboards is here.

Working Paper 21 – “Executive Briefing Book”  (June 2018)

The following table represents the metadata regarding the data released for the states. (There are other sources of trade data not included in this study.)

The above graphic shows some metadata about the data sets I integrated into Tableau for the ITTS member states.

In reviewing the project, I talked about the need for the client to understands how to access the data and the data elements are included, so they can duplicate/review the work. I also discussed how to organize the data into Tableau, including challenges I faced regarding one to many joins and matching geography.

If you are interested in more discussions on trade data, I did a presentation at the University of Southern Mississippi regarding trade brochures developed for the ITTS member states (there are some other trade reports posted at http://www.ittsresearch.org/state-reports.html).

The real takeaway from all this…trade data is complicated, especially if one forgets the metadata, for surely the devil is in the details.

Posted on

How Does Intelligence Fill Santa’s Sack?

Based on my earlier blog post, that intelligence is the information that one uses as the basis of wisdom, I would add yet another examination of Santa and Kringle Inc’s use of big data and system processes. 

So, I start with the obvious, the goal of Kringle Inc., namely, to determine what gift goes to what child in what location on a certain day.   So, when Santa talks about his busy day, he is managing a lot of information so that he can manage many items, ranging from sourcing, production, trade and duties, and personnel matters, from reindeer training to elf productivity.  In all cases, the clock is ticking (I wonder if Santa screams like Tom Hanks in Cast away). 

We know that Santa has list concerning a child’s conditional status concerning a child’s ranking! The child, based on the behavior (crying, asleep, good, and bad), is collected from various sources, such as parents, teachers, mail, (Ray Steven’s song suggests it may borderline on obsessive).  Given the times, I wonder if Santa purchases search and social media information, or aggregated email services. So, if the child passes  “the Good Test”, the next item is what should a child should receive.  Here Santa evaluates the child’s gift(s) based on preferences, previous year’s gifts, age, etc.   In many ways, Kringle Inc. maintains several relational databases of each child’s demographic (location, age, etc. and preferences) that is joined to another database on gifts. 

These gifts are then placed into a production stream, employing many elves. (Kringle Inc. successfully hires and trains workers in a rural community, and that is worth something!)  The toys are loaded into the sled, and after negotiating all the appropriate tariffs and import fees, Santa is on his way.  So, Kringle Inc. has to maintain inventory, customs filings, to translate the child gift to the location and delivery window.  I wonder how Kringle Inc. manages to satisfy animal health codes, but that is simply a minor problem considering all the other transportation questions (hazmat cargoes) Kringle Inc. must organize in the sled’s stowing plan.

So, we have the Kringle Inc. decision tree.

But not every gift from Santa goes through this process.  Sometimes Santa shows up (yes this is me in 1984) and hands out toys at a neighbor’s day care.  (And only in Louisiana could a Santa Claus wear rubber boots.)

  • bruce lambert playing santa clause in 1984

Merry Christmas to all,  Bruce Lambert


Posted on

Tell Me What You Want


When I was younger I read the story of the three vinegar tasters in “The Tao of Pooh“.   What does vinegar taste like maybe a question for a cooking class, but as a researcher, its relevance is more important as “why do people need this information”.    Everyone who is asking a question does so for many reasons, but they can be grouped into some very large clusters:

  • information to impress (everyone wants HUGE numbers),
  • wisdom to inform a decision maker (help make a decision), or
  • to satisfy a program requirement (support a decision already made).

As a researcher, people come and ask you for a question to be answered.  The challenge is you may have to help them ask the correct question, which sometimes they may not understand why formulating their request remains a critical step for a successful study.  In the short story, “Ask A Foolish Question”, by Robert Sheckley, there is a machine capable of answering any question, named the “Answerer”. The problem is that people asking do not know as much as the computer (or in some cases, the researcher’s knowledge on the topic).  The Questioner must know something about the answer for the Answerer to provide the correct information.  In some ways, every researcher must learn how to pass on information, but they must also inform the people asking the question to help them both format their question and understand the answer.  (Richard Feynmn  would argue if you can’t explain it to a five-year old, do you really understand the topic?)  Sometimes people feel like Ralph where there is too much ambiguity to even phrase the question.

In helping people frame their research, often we simply have to listen, asking them what they want and how will they use the information.  Oftentimes this becomes a collaborative process between the answerer and the asker.

With the advent of the internet, many assume the information they seek is readily available, often ignoring the work and effort it takes to transform data into something useful,  This failure to understand could lead to the discounting of the work associated with exploring the question, assisting in organizing the research, and providing a satisfying answer. So, please a little patience goes a long way for everyone to agree on how vinegar should taste.

Posted on

Buying a Cup of Coffee – Data Becomes Wisdom

The Ancient Mariner could have as easily said “Data, Data, everywhere”,  as discussions regarding big data and other analytical approaches seem to be the order of the day Wall Street Journal.  We often make data out to be this mysterious element, but data requires context to be useful to anyone if they intend to make a decision, as I hope to show below.  For example, data becomes information when it is categorized, which then becomes intelligence when a decision can be made, and ultimately wisdom when an action occurs based on intelligence.

One of my favorite quotes about Coffee comes from Charles Maurice de Talleyrand-Périgor, who said that the perfect cup of coffee should be: “Black as the devil, hot as hell, pure as an angel, sweet as love.”  I would agree that a good cup of coffee is a bargain at any price, so let’s think about how the decision to buy a cup of coffee can explain transforming data into wisdom.

The objective:  I would like to purchase a cup of coffee at Giddyup Coffee in Folsom, LA.  So, I asked the two baristas if I could do take a few pictures for this blog post.  They thought it funny that someone would actually do this, but they agreed.  So, the research question is: do I have enough loose change to purchase a cup of coffee.

 

Data:

Loose Coins – represent Data

There will be a cost of purchasing a cup of coffee, so I look into my coin purse.  These loose coins are simply data points, each representing a certain value.  (Coins contain other information, such as their size, year and place it was minted, as well as the coins condition based on circulation.  These data points are not relevant for this purpose are ignored.) However, beyond knowing that I have coins, I do not know if these coins are enough to purchase a cup of coffee.

 

 

 

 

 

Information:  The loose coins now need to be organized before I can actually make a decision, so the coins were put into different categories.  This act of putting the coins into categories, based on the relative value of the coins, resulted in the data about the coins becoming information.

Data becomes Information

Intelligence:  Now that I know the relative value of the coins, I next have to make a comparison.  Do I have enough to actually purchase the coffee with the coins that I have?  So, I add a new data element, namely the posted value of a cup of coffee.  So, the addition of the information posted on the menu allowed me to determine if I could make a purchase with my loose change.

Wisdom:  I bought the cup of coffee, once I had enough intelligence to make a decision based on the cost of the cup of coffee and my loose change.  (Wisdom is the only attribute with a future component, namely, data, information, intelligence are all static elements at the moment a decision is made, but Wisdom will influence my future actions.)

 

I learned this paradigm as the more formal DIKW: Data, Information, Knowledge, Wisdom.  While one could argue that the DIKW is based upon filtering data to make a decision, I changed Knowledge into Intelligence.  I see Knowledge represents a broad body of information, based on many factors, including not only the data itself but the cultural, contextual relationship of the researcher to the topic being researched.  For example, I could have taken the coins to a Coinstar or a bank, or made a different decision concerning these coins.  For my father who abhors coffee (his loss), the research question (can I purchase a cup of coffee) would mean nothing to him,  (much like saying “what does that have to do with the price of tea in China?”!)  For me, knowledge serves not as a filter of the data/information as normally discussed in the DIKW paradigm, but rather a filter through the act of transforming data to wisdom can even occur.

Finally, we can not remove the researcher from the research, but a good researcher should understand what data elements are useful to become transformed into intelligence based on understanding what answer is required.

Here is a toast to your health!