Developer Showcase: Multilingual Article Summary Generator

Scarcity and abundance, both situations can be a good problem statement to address. Take the case of Information. Before the Internet era, Information about anything was difficult to find and scarcely available. Now with the proliferation of world wide web, there is an abundance of information on almost any topic. It is even easier to publish content that adds to the existing information knowledge base. As a result, we are dealing with the abundance situation. How about solving this problem with an article summary generator tool that spews out a byte-sized summary of any online content in your preferred language.

Tackling the Information Abundance Problem

This idea can be envisaged as a web app that generates a summary text out of the information. This helps us to consume information in byte-sized summary text nuggets so that we can sift through loads of content without wasting much time. 

Let’s build the app by leveraging the Rakuten RapidAPI’s API catalog. We will use the AYLIEN Text Analysis API for this purpose. We also want to add a bit of personalization feature so that users of this app can receive the summary text in their preferred language. For this, we have chosen a translation API.

If you haven’t done already then you should sign up for Rakuten Rapid API to get access to these APIs.

Rakuten RapidAPI is the world’s largest API marketplace with 8,000+ third-party APIs and used by over 500,000 active developers. We enable developers to build transformative apps through the power of APIs. Find, test and connect to all the APIs you need in one place!

Check out some of the world’s best APIs including Microsoft, Sendgrid, Crunchbase, and Skyscanner.

An App to Generate Summary Text from Online Articles

Imagine that we are looking for some information on President Donald Trump, and we stumble upon his Wikipedia page. Instead of reading the whole article we could generate a summary text page out of the article content. This is the article summary generator app in action.    

Although this is a very brief summary, we can choose to go deeper to extract more information.

So let’s get started and build a small web app to generate multilingual article summary information from a URL. We will call it the “Multilingual Article Summary Generator”. The source code for this app is available here. You should download the code and follow along with the steps below to build this app.

Prerequisites

Before you start, refer to the README file for the installation instructions to update the code with your API keys and install the dependency packages for building your own version from the multilingual article summary generator.

Additionally, you must have Node.JS installed on the computer where you want to test this app.

Step 1: Build the app UI

The app UI is a simple Bootstrap interface defined in multilingual-article/public/index.html

As evident from the UI, the app displays a form that asks the users to enter the URL of the article that they wish to summarize.

Upon submit, the form calls /generate API to extract summary text from the URL.

Step 2: Implement /generate API

This app uses the express framework for defining the URL routes. This is defined in multilingual-article/index.js

Apart from the ‘/’ route which displays the main application, we have defined ‘/generate’ route. This is the endpoint that triggers the internal logic of the article summary generator and returns the summary text.  

Step 3: Connect with AYLIEN Text Analysis API

The ‘/generate’ handler makes a call to the AYLEN Text Analysis API to extract &  summarize the content pointed to by the URL. The API returns an array of strings each containing a summary text sentence.

app.post("/generate", (req, res) => {
  let unirest = require('unirest')
  //unirest.get("https://aylien-text.p.rapidapi.com/extract?url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBatman")
  unirest.get("https://aylien-text.p.rapidapi.com/extract?url=" + req.body.url1)
    .header("X-RapidAPI-Key", rakutenAPIKey)
    .end(function(result) {
      //console.log(result.status, result.headers, result.body);

      let title = result.body.title

      unirest.get("https://aylien-text.p.rapidapi.com/summarize?url=" + req.body.url1 + "&mode=short&sentences_number=1")
        .header("X-RapidAPI-Key", rakutenAPIKey)
        .end(function(result) {
          let sentences = Array(3)
          sentences[0] = result.body.sentences[0]

Step 4: Translate the Summary Text using Translate API

Now that we have got the summary text, the next step is to translate it. To keep things simple, we have only considered the first summary text sentence. Also, instead of offering the user to choose a language, we have translated the summary text into two preconfigured languages, namely Japanese and Spanish.

Here is how we call the translation API to convert the summary text.

unirest.post("https://translator.p.rapidapi.com/api/translate")
      .header("X-RapidAPI-Key", rakutenAPIKey)
      .header("Content-Type", "application/x-www-form-urlencoded")
      .send("input=" + sentences[0])
      .send("target=ja")
      .end(function (result) {
              sentences[1] = result.body.ouput
              //console.log(result.body);
              console.log("+++ SENTENCE  +++");
              console.log(sentences[0]);
              console.log("+++ TRANSLATE RESULT : ja +++");
              console.log(result.body);
              unirest.post("https://translator.p.rapidapi.com/api/translate")
      .header("X-RapidAPI-Key", rakutenAPIKey)
      .header("Content-Type", "application/x-www-form-urlencoded")
      .send("input=" + sentences[0])
      .send("target=es")
      .end(function (result) {
                  sentences[2] = result.body.ouput
                  console.log("+++ TRANSLATE RESULT : es +++");
              	  console.log(result.body);

There are two calls to the API with “target=ja” and “target=es”, for translating to Japanese and Spanish respectively.

Step 4: Display Result

Once the API call returns, we will have the summary text in English, Japanese and Spanish.

Now is the time to capture the summary and display it in the UI.

For this, we have used the PUG template engine (/public/results.pug). The summary text along with its translated versions are passed to the PUG template engine to generate the HTML code.  

The PUG template is self-explanatory. It formats the summary text into unordered lists.

body(style="margin:8px;")
  
    ul(class="list-group")
      li(class="list-group-item active") #{title}
      each sentence in sentences
        li(class="list-group-item") #{sentence}
    
    a(class="btn btn-primary" href="/" role="button") Go Back

What’s Next? Article Summarization on Steroids

This app was just a small showcase of the possibilities of using text analysis APIs for generating summary text out of a huge volume of content with the power of NLP. As we know, the Internet is a linked medium, wherein all web resources have a way of linking to others via hyperlinks. If we enhance this app to follow hyperlinks, then we can envisage an automated auricle summary generator for thousands of web pages.

Let us know what you think about this idea. You can explore the other options in AYLIEN text analysis API to extract more useful information from URLs and build some interesting features such as paraphrase maker.  You can explore more text analysis APIs from the Rakuten’s website. Ee can’t wait to see what exciting features you can add to make this the best summary generator app.