Lima Vallantin
Wilame
Marketing Data scientist and Master's student interested in everything concerning Data, Text Mining, and Natural Language Processing. Currently speaking Brazilian Portuguese, French, English, and a tiiiiiiiiny bit of German. Want to connect? Send me a message. Want to know more? Visit the about page.

Contents

Don't forget to share:

Share on linkedin
Share on twitter
Share on facebook

Don't forget to share:

Share on linkedin
Share on twitter
Share on whatsapp
Share on facebook

Natural language generation is one of these trendy topics that everyone talks about as being something more complex than it really is. We see free associations of the term to deep learning in a way that we are forced to believe that this is the only way to do it. Or that it’s only used to build chatbots.

It is not.

What’s Natural Language Generation?

Text generation is as old as Eliza​1​, an early natural language processing program created in 1964. The application was able to produce text et to emulate a conversation using pattern matching and substitution methodology.

The computer didn’t really understand what it was writing. Nor yesterday, not today.

NLG, for short, could be something as simple as a text spinner. Or as elaborate as hugging face transformers. At the end of the day, the only difference between the latter and the former is that the first relies on a more strict set of rules to work while the second will try to build text-based on statistical processes.

Saying that you use deep learning will also give you extra points if you want to look trendy.

Just kidding. Deep learning text generation is more rich and diverse, but it’s also more expensive and, let’s be honest, not always needed.

Let one thing be clear: text generation does not imply that the machine has a superior level of intelligence when it’s produced using machine learning models or artificial intelligence.

In fact, as I just stated before, deep learning is only a probabilistic way to create text, while pattern filling depends more on human intervention in order to create the first templates.

When should you use Natural Language Generators?

Again, let’s take a look at my company’s, Mad Affinity, case. We use text templates to generate product descriptions based on the products’ attributes. It’s merely a matter of merging a word with a template. But it works beautifully for us.

It could also work well for you and for many other companies.

NLG based on templates and on pattern substitution still empowers from newsletters to chatbots all over the world. Business usage of this kind of technology dates from 1980, with WordStar​2​, an application that was able to merge addresses and mailing labels.

This may seem sooo “1980”. And yet, it’s what marketers all over the world do when they create email campaigns.

This simple mechanism allows the generation of simple texts, which could be used from push campaigns to urgent natural disasters SMSs. In fact, it’s is already used.

Imagine that you want to create a Twitter account that writes a new tweet every time that weather changes in a certain region of your country. Rather than spending the whole day checking the changes on some weather website, you can create templates and automatically fill the gaps with information retrieved from an API.

For example:

import random  

temperature = "74" #get it from some random temperature service API 
local       = "New York" #the local that you are using 

# choose a sentence from the template pool 
templates = ["Current temperature in {local} is {temperature}F.",
             "Here's the current temperature for {local}: {temperature}F!'",
             "Thinking about leaving? It's {temperature}F right now in {local}.",
             "Temperature update in {local}: it's {temperature}F right now."] 

# pick one and print 
sentence = random.choice(templates).format(local=local, temperature=temperature) 
print(sentence.format(local, temperature))
#possible outcome
"Thinking about leaving? It's 74F right now in New York." 

You could create as many sentences as you want and just “glue” the temperature and the place to them. By the way, congratulations, you’ve just used natural language generation.

Using NLG: should I choose text spinners or deep learning?

When it comes to the technological side of the discussion, one should keep in mind that text spinners will solve most of your problems of content production on eCommerce or in sites that make use of repetitive text.

Product descriptions could be easily generated only by getting the product’s attributes and transforming them into full sentences. Instead of listing a product’s colors, you can expand this into a longer sentence using the tone of your brand.

This model is a quick and easy way to create content for your website using data that you already own. But you can also use this to generate automated intelligence reports using analytics data.

More sophisticated technologies are useful when you need to create complex texts, like summaries. However, text generation based on pre-trained models is not exactly reliable and some human work would still be needed to polish the whole thing.

Deep learning can be useful if you want to summarize text to use it as a basis for new content. The problem is that most of us mortals don’t own powerful enough machines to quickly generate summaries of long texts.

In a nutshell, templates are useful when you need to generate simple sentences in an extremely controlled way. Deep learning is useful if you want to generate complex sentences from scratch.

What’s really important when using NLG?

More than the technology itself, it’s important to think about the process. According to Dale and Reiter​3​, the typical stages of NLG are:

  • Content determination: planning the information that will appear in the text. In our example, deciding on using temperature and local.
  • Content structuring: planning on how this information will appear. For instance, should we use the temperature before local or vice-versa?
  • Aggregation: write and merge similar sentences so the reading flow will look natural for the reader.
  • Lexical: deciding on the word usage and on the concepts that will be used in the text and justifying this choice.
  • Referring expression generation: decide on how to make references to objects, names, and concepts already quoted in the text. For instance, we could use “the Big Apple”, “Manhattan”, “the city” to refer to “New York”.
  • Realization: this is where you write the actual text, taking into account syntax, morphology, and orthography. The final text must be grammatically correct and the tone should like human.

Complex logic can and should be used when using text spinners and templates. In our temperature example, we could define different sentences according to the actual temperature. If it’s raining, we could create a set of messages reminding the user to bring an umbrella.

Considering anaphoras is also extremely important when building text. Your spinner should take into account the use of expressions whose interpretation depends upon another expression in context and provide solutions for cases where we need to add, remove or replace tokens such as “in New York”, “in the USA”, “at 55 Oxford Street” or “on the Hudson River”.

Controlled natural languages and text generation

The notion of controlled natural languages (CNLs) is well known in the translation field. A CNL is a subset of a natural language, such as French or English, that’s obtained by reduction and restriction of certain grammar rules and vocabulary in order to reduce or eliminate ambiguity and complexity.

Controlled natural languages are used, for instance, by big companies that produce thousands of pieces of content by different employees. In order to keep a certain unity, rules are introduced to simplify and give consistency to the contents.

In addition to adding tone consistency, CNLs (or technical languages) can also improve readability and enable automatic semantic analysis.

Creating a technical language may improve not only machine translation (consequently decreasing the cost of translating content to another languages). It’s also a great way to generate templates that can be used later by text spinners. CNLs restrict the writer by creating sets of instructions of how something should be written.

In general, the instructions can be something such as “keep sentences short”, “avoid passive voice”, or “use glossary-approved words”. But you can use this general idea to create powerful sets of templates that could be used to leverage text generation.

CNLs are not only useful for creating templates, but also for training machine learning models and for improving automatic translation. As language is ambiguous, model trained on a controlled language don’t have to deal with the ambiguity, anaphoras and other or with complex grammar rules.

On Controlled Language Optimized for Uniform Translation (CLOUT)​4​, Muegge describes a set of rules that could be used to create to reduce ambiguity and create a CNL, such as:

  1. Write sentences that are shorter than 25 words.
  2. Be brief.
  3. Avoid subordination.
  4. Write one sentence per idea.
  5. Avoid gerunds.
  6. Repeat sentences that express the same content.
  7. Work with repetitions and avoid synonyms.
  8. Write grammatically and orthographically complete sentences.
  9. Use the active form.
  10. Repeat the noun instead of using pronouns.
  11. Use articles to make it easier to identify nouns.
  12. Do not use technical words.

Bottomline here is to use simple phrasal structures to improve readability and usability. It also helps to create a a structured language ready to be used with pattern identification.

However, some rules of CNL creation seem to go against the aggregation rule, which says that sentences should be merged and pronominalization used. Well, a technical language is only one of the many ways that you can find to simplify the task of generating natural text. But it’s not the only one.

NLG doesn’t have to be hard or fancy

I am the kind of person who likes to discover what’s and trendy, but I usually recommend other people to stay away from what shines until it’s available to the average Joe.

Deep learning is great, beautiful, and impressive, but it doesn’t suit everyone. There are simpler ways to start generating content that could do wonders for any business with a programmer with a basic knowledge of Python.

Have you ever considered this alternative? Have you already tried using Natural Language Generation in your company or for a project? How was it?

References

  1. 1.
    ELIZA. Wikipedia. Accessed December 15, 2020. https://en.wikipedia.org/wiki/ELIZA
  2. 2.
    WordStar Training Guide . Bitsavers. Published February 1983. Accessed December 15, 2020. http://www.bitsavers.org/pdf/microPro/Wordstar_3.3/Wordstar_Training_Guide_2ed_Feb83.pdf
  3. 3.
    Dale R, Reiter E. Building Natural Language Generation Systems. Cambridge University Press; 2000.
  4. 4.
    Muegge U. Controlled Language Optimized for Uniform Translation (CLOUT). unpublished. Published online 2002:2. Accessed December 16, 2020. https://works.bepress.com/uwe_muegge/88/download/

Don't forget to share:

Share on linkedin
Share on twitter
Share on whatsapp
Share on facebook

Leave a Reply