An application to learn English from YouTube content
There are many ways for people to acquire language skills beyond structured classes in today’s digital world. Digital channels offer benefits such as flexibility on when to learn, lower cost, and more up to date and engaging content formats. Videos specifically, can be more engaging than written text, and personalised content means more interesting topics are shown.
We thought of leveraging the YouTube platform’s massive content database, subtitling features, and personalised recommendation engine. YouTube is already a great product. Is it possible to bootstrap a solution and do better? Our team not only undertook this challenge but we did it at Junction Tokyo 2019 Hackathon as part of the (read about it here). We ended up winning both the ‘API Hack’ sponsored by Rakuten RapidAPI and the Grand Prize. Here’s how we did it.
Many people already watch Youtube to learn English. It’s an amazing free English textbook but underutilised. Many users don’t even leverage subtitling on YouTube clips. Translation offered a way for us to improve user experience since it’s currently very inconvenient to look a word from a script that you don’t understand in a dictionary which language apps don’t usually offer. Users tend to refer to Google Translate. This requires multiple copy/paste actions which disrupts the user’s learning experience. We set out to create a YouTube app specialized in Learning English by combining YouTube video transcripts and a dictionary.
We think our solution fills a gap in between YouTube itself and language-learning apps. YouTube provides great English language video transcripts and English-learning apps don’t often have a built-in dictionary feature.
Rakuten RapidAPI turned out to be a great platform for finding all your favorite APIs. Managing external APIs are usually quite difficult since you have to navigate multiple different providers. With the Rakuten RapidAPI platform, you can manage and see the statistics of all your APIs. The test endpoints feature is also amazing. We powered this application using Microsoft Text Translation API, Twinword Word Dictionary API, and OneSignal API.
Editor’s Note
Team ‘Hack English with YouTube’ won the ‘API Hack’ track AND Golden Ticket at Junction Tokyo 2019. The hackathon challenged participants to build solutions and applications for real-world problems by leveraging APIs to push the development boundaries. Check out full finalist pitches live streamed here.
‘Hack English with YouTube’ not only tapped into YouTube users, but did so beautifully with an impressive testflight app which they completed over a weekend. Finally, we saw tremendous potential in this becoming a language learning platform with very little additional work.
Interested in using Rakuten RapidAPI to power your next hackathon? Contact us.
User Journey
Our application built on top of the YouTube platform’s content, subtitling and recommendation engine. We added a smart dictionary function and built a custom UI that mimicked the look and feel of the YouTube mobile app.
In order to get this development done within 36 hours, we leveraged several APIs from the Rakuten RapidAPI platform. This saved us time and in a real-world setting, makes service provisioning affordable.
Content Discovery & Curation
Users first select their interest areas to receive curated content from YouTube’s video database. They then bookmark their favorite videos which become the basis of their lesson plan. Video content was available via the YouTube API.
Consume Content & Learn
Each video a user watches will be populated with an English video transcript via Google API. Word definitions and example uses pop up when a user double-taps on the words they’re unfamiliar with. This eliminates manual copy/paste actions that require users to navigate out of our app. This dictionary function was powered by Twinword’s Word Dictionary API.
Selected words also integrated pronunciation featured which was provisioned using the BING Speech API. Microsoft’s Text-to-Speech endpoint could then be used to read out a particular word.
Smart Dictionary for Reinforced Learning
Words that a user looks up are automatically added to a ‘word list’. We help user’s learn better by periodically prompting the, to review words that they had previously looked up. This is personalized, curate word list becomes the basis of our user ‘Smart Dictionary’.
We use device push notifications via OneSignal API to achieve this. Developing a push notification service from scratch requires implementation of a specific server, utilizing push notifications protocols, and a corresponding API. Using a public APIs let us send push notifications out of the box, enabling us to achieve this in minutes instead of days.
Tech stack
Our backend stack is composed of a CentOS server hosted on AWS. The backend language used is PHP using the PhalconPHP framework. We used MariaDB to store data and containerized the backend with Docker for scalability.
Our front end is an iOS application developed in Swift. We used CocoaPods for dependency management with popular libraries like Alamofire for the networking and Kingfisher for the image caching.
We used several APIs to power our application. These provided high quality, cost effective content, data and functionality out-of-the-box.
- OneSignal API: Send push notifications as prompts from user smart dictionary to reinforce learning.
- Twinword Word Dictionary API: Look up a chosen word and provide a definition and examples.
- Microsoft Bing Text to Speech API: Read aloud a word to train users on pronunciation.
- YouTube API: Grab youtube video data for content feeds.
- Google API: Grab YouTube video transcripts
Next Steps
Our team plans to take this from prototype to live service. We’ve already identified a large number of improvements needed to make launch possible. For example, we would like to implement deep learning features and upgrade our smart dictionary. If we can do this, we believe that we can help hundreds of millions of non-English native speakers improve their language skills with engaging content and in an easy way.
Please reach out to Jeon by email if you have questions or suggestions (jeon@avanssion.com).
Featured Developers
Daiken Jeon (aka Haneal Jeon) comes from Korea and has over a decade leadership experience in art direction and management in Tokyo. Daiken specializes in service building by bringing together expertise in humanities and cutting-edge marketing strategy. His direction breathes a new life into a blank, that is every beginning and end in his team.
Daiken created Avanssion in 2018 with Florian and Kuma with the ambition to build their own start-up. His willingness to embrace challenge never fails to motivate his team members. His journey has just begun.
Facebook | Twitter | LinkedIn | Github
Florian Ludot is an iOS engineer and technology enthusiast from France who is currently based in Tokyo. As a passionate and ambitious engineer, Florian walked away from an offer to join Air France and chose to learn Swift instead in 2016.
He met Daiken in 2016 and joined Avanssion as lead engineer after relocating to Japan in 2018. He achieved the highest level of iOS engineer in just 1 year.
Facebook | Twitter | LinkedIn | Github
Kuma Jeon (aka Deok-young Jeon) is a veteran Korean engineer. In his 13 years of experience, he has mastered 8 programming languages and 6 database technologies. His systems are in use by 12 universities and 5 government offices in Korea.
Kuma first worked with Daiken in Japan in 2012 and completed 5 projects. He’s a consummate team player who supports those around him instead of standing out. He’s obsessed with optimized databases and creating amazing systems that are absurdly stable and effective. Even if Team Avanssion has a great plan or skills, nothing could be a reality without him.
Facebook | Twitter | LinkedIn | Github
Rakuten RapidAPI is the world’s largest API marketplace with 8,000+ APIs and used by over 500,000 developers around the world. We enable developers to build transformative apps through the power of APIs. Find, test and connect to all the APIs you need in one place!
Check out some of the world’s best APIs including Microsoft, Sendgrid, Crunchbase and Skyscanner.
Leave a Reply