The Best Search Engine For Hindi Film Songs ?

One key challenge with searching for hindi film songs is that most song titles (names) are spelled / recorded in roman english. Try Apple Music or YouTube or Gaana and their databases are full of badly spelled, differently spelled english based renditions of hindi / urdu words. It is no surprise that it is very hard to find the song you are looking for coz the song may be spelled with a different spelling than the one you have in mind. For example, there are hundreds of songs that start with “मैं” (me) but they could be spelled with “mein”, “main”, “mayn” or even “men”. More intricate words have even harder spelling issues.

Moreover when user types 3-4 words of a song, which is the dominant way to search, the probably of one of the words spelled differently increases which reduces the probability of finding the exact song! So the more you type the poorer results you get!

The reason for this mess is the inherited nature of language computing. However with rise of smartphones we are seeing rise of language usage and language keyboards on the smartphones are making it easier to type in indian languages. So why do the music vendors continue to support english based song names /titles? Can we not challenge this?

“Filmi Filmy” is our app where user can view song names in Hindi. Just choose the language setting and the app will automatically translate the song name and show you the Hindi song title. This itself is hugely useful because beyond the 80-100 mi users, most users cant even read english names. So having hindi song displayed in hindi is a god-send.

To improve the search experience in “Filmi Filmy” we added Phonetic Search capability/. We implemented the “soundex algorithm” of Phonetic Search and found that the results are better, however we were still not satisfied. The soundex for hindi sounds is clearly also evolving. E.g. there exists a song titled “Lagi Aaj Sawan” and when user searches for exact phrase, it works, however when user searches for “Lagi Aaj Savan”, even the soundex result is fails to match it.

Then we experimented with “Edit Distance” based search – how far is the search string from the song name? The results were better, but still not satisfying. E.g. user searches for “आप” because user wants to search for song starting with this word, however user will also get results like “आ छुप जाएँ सनम” in beginning because both the letters of the word are found in this song. Definitely not a solution when you want to build THE BEST SEARCH ENGINE for hindi film songs. Period.

That is when we realized that our search which was implemented on complete text of song, needs to be implemented on individual words and then we have to combine word results to make complete Title result to make better. We have now evolved our own algorithm that uses a combination of phonetics, edit search and semantic analysis to detect the key words in the search string and find the same in the song database. Instead of doing a search on the full name, the engine searches for the key words/phrases and then combines the word results to make complete Title result. Further, we use a weightage scheme to prioritize the keywords depending on frequency of appearance in the database and position in the search string. This approach improves the search results appreciably. And then if you add character level Phonetics with Edit Distance, the results are almost near perfect search! E.g. ख़्वाहिशें and ख्वाहिशें are considered exact match instead of approximate match.

There are still certain challenges to be worked out. Text in Hindi can be written using different Unicode characters(!), which in turn makes our algorithm weak in cases where the word looks the same but internally they are different Unicode characters and thus not a match. Also there are challenges where a word can be written differently (e.g. जिंदगी vs ज़िन्दगी).

But inspite of these challenges, we now have, we believe, the BEST SEARCH ENGINE for hindi film songs. The new search will be available in an updated release of the app “Filmi Filmy” soon. Do try it and share your feedback. And if you are frustrated by the search experience on Apple Music, YouTube, Gaana etc do send them a note to license our technology!!

Your Daily News Summary @ 9 pm

In my post dated October 25, 2015 (http://www.anveshanblog.com/the-big-news-of-the-day/) I talked about the idea of push notification in a news app. Should the app be chatty and send notifications every hour? Should the app provide personalized schedule of notifications that the user can control?

We came to the conclusion that a better option may be to offer a summary of the day notification say at 9 pm that provides the top 30-50 news stories of the day and a chance to browse, preview and read in detail. Here is what I wrote in October – “Consumers are caught in their daily grinds and are juggling a barrage of work, personal and social messages. But all consumers want to be aware of their world, the news that matters to them. What better than to send them a capsule of the top news of the day and provide a quick way to catch up on the BIG news with the means to research any specific story in as much depth as possible. After all this is what the 9 pm TV news is all about – top stories of the day; 1 min, 60 stories; headlines of the day are all about giving you the top news as curated by the editorial team in a quick 5-minute recap and then debate the top 3 issues over next 1 hour.”

We set ourselves a 1 week target for this feature. However, when we started building this we realized that there was a major point of contention between the core feature of the app, ie the BIG news NOW and this new feature “summary of the day @ 9pm”. How do we integrate two major ideas into a single product? Some folks on the team argued that we should drop the hourly BIG news feature and simply concentrate on the “Summary @9″feature. They went so far as to suggest even renaming the app!

The others argued that users want to be able to check the BIG news at any given time AND ALSO see the summary news @9. So we must offer both options. But how to do this? What would be the most predictable way to accomplish this? We have struggled with many ideas, different layouts, different themes, different colors etc.

Our solution? Use the notification handler and the time of day to change the application layout for the “special” Summary @9 feature. So when you get the app, you will notice that at 9pm the app suddenly, visibly morphs to a new layout – the Summary layout. We want to tell you, the user that the summary of the day is important and in fact so important that we are ready to change the app layout at 9 pm. From 9 pm to 11 pm, the app follows the summary layout.

If you miss the summary notification and/or miss the summary time-window, you can ask the app the show you the previous day’s summary. So, there is an automatic summary view and also an on-demand summary view. We want to ensure that you never the miss the day’s summary from 9pm on the same day till 9 pm the next day.

Check out the screen shots of the “Summary @9” and do try the updated app. You will get a notification at 9pm with the top story for the day and count of other summary stories. Slide the notification and the app will open in the summary view. Open the app directly and the app will morph the view to the summary view. Missed the summary-window? Tap on the summary icon and the summary view with last day’s summary news will be shown. Read the summary stories, check out the related stories and gorge on the detailed stories. All of the in-depth reading is maintained.

Screenshot_20160314-214429-168x300

Do try the app and the “News of the Day Summary”.

Android Play Store Link : https://play.google.com/store/apps/details?id=com.plabs.apps.innews&hl=en

A Personalized Song List For You

Our app “FilmiFilmy” was the first to recognize that there was a treasure of hindi film music on YouTube but the challenges in discovering songs was a big hurdle to use. We aggregated the songs, converted the roman song titles to hidni (the first app in the world to show hindi song titles) using some cool algorithmic tricks and also added a hindi voice search (the first app in the world to support hindi voice search). This week, I have heard that YouTube Music is trying to do many of the things we have already done in the app. Its a great vindication of our idea but also a threat to see a big gorilla come into this space.

So how do we stay ahead of YouTube? Or force them to consider a partnership given our innovation is significantly ahead of them?

The solution lies in the magic keyword of internet “personalization”. FilmiFilmy currently has zero personalization. It treats all users as “same”. It is a smart catalog aggregator with a smart hindi UI and hindi search. It has some tools to select your favorite songs into a playlist and share them but this is quite generic.

What should be happening is that we should “know” our users and their tastes. Since have FB login, we should be using this to build a persona of the user based on their FB profile, likes/dislikes especially in music. We should be tracking what they have searched, viewed, listened to, shared, selected in the app. We should be tracking what they have shared on their social profiles. We should be using this to build a model of their musical interests.

The profile of their musical interests should be used to create a song feed that is relevant and personalized for the user. When they start the app, the app can start from their last search, the last song shared/played, the song last commented on facebook (sort of like re-targeting) or from the cohort of like minded people – what the social vector is listening to. Or the top song played on the platform in the last 24 hours or similar time-period. All of this should be managed/configured by the user through a preferences mechanism. The user declares their choices/priorities and the platform uses their engagements on the app and outside in the public domain to meet their needs. The existing catalog should get reduced to a search function, at best.

This seems like a cool idea. But is it only limited to our app? music? Or is this a recipe for a broader app-personalization technology that all consumer apps should be using – whether the app is music, news, books, articles or shopping or ordering food, tickets, concerts – anyplace where a consumer persona helps an app provide better service and smart assistance is game for this technology. Wait, isnt this was Cortana, Siri are all about? Becoming the smart digital assistant to the user. Seems like there is validation of the idea!

What happens after the user is presented the first song / recommended song? If you get it right, the user plays the video, listens to the song and is ready to accept the next recommendation. If you get it right, the user browses onto the next recommendation and if the next recommendation is not relevant, switches to search to find a more relevant starting point. I could live with that!

So, the ideas are in. Now to build this capability. And perhaps find some partners who can pitch in to fund the development of a larger persona and interest database that everybody can use for recommendations. Now that will take some doing.

BTW, if you look at the previous post, you will see that I promised to build the feature to pull images from the cloud into our gifting app, ThotOfU. Happy to confirm that both iOS and Android versions have now fully functional ability to use any of your own pictures for use in a greeting card. Try it and see how simple it is to pick an image from your flickr / instagram account and create a greeting card with it.

A Greeting Card With Your Own Pictures

100s of cards are sent weekly thru our personalized card designer app “ThotOfU” across 40 countries. Many users have asked for the ability to use one of “their own” pictures in the cards in addition to the image gallery we provide.

We think this is a great idea. Imagine sending a birthday card to a friend which has you and him in a nice setting. The pictures brings happy memories of time spent together and brightens someone on their special day. Or a picture of a vacation used in an anniversary card – if a picture tells a tale, clearly there is power in using your own pictures to make custom cards.

Where do people store their images? Many of us, like me have 1000s of pictures on our phone, cloud storage services such as Dropbox, iCloud, Google Driver and picture sharing sites such as Flickr, Instagram, Pinterest etc. Wouldn’t it be great to offer a way for people to brose their picture libraries via the app, pick the picture they like and use that picture in the card?

We are delighted to announce the latest update to ThotOfU – the ability to pick your own pictures from your device and/or the cloud storage and/or the picture sharing sites for a small convenience fee (paid via IAP credits). Just download the latest app, pick the template you want, choose the source of your pictures (you must have IAP credits to do so), pick your picture and watch it appear in your card, type your caption/message and send the card via WhatsApp, Facebook or even email.

thoutofu1thoutofu2

The BIG News Of The Day

With “inNews”, our news aggregation app, we have brought 3 key ideas to mobile news readers:

1. Users don’t need news served via feeds & categories, they want to know whats the top news at any time. Technology should determine what is important based on social, readership, influencer, velocity, volume of content on the story.

2. Users prefer both short-form and long-form content. Short form content can be curated manually or generated through automation.

3. Technology should provide a 360-degree view of the news story and all media associated with it – all related stories, images, videos and a historical context so readers can understand news better

Our initial analysis with first 500 users shows that while there is good engagement with the content, there is not enough engagement with the app. The app currently relies on the user to open / launch it to see the top news and the associated 360 deg views. We are not giving any reason to open the app.

In an earlier post, I have talked about notifications and what sort of notification one should have. The easiest way would be to send a notification each hour with the TOP news at this hour. In the age of chatty apps, a user gets 5-10 notifications per app and with 40-50 apps, this could itself become an over-kill. Another way would be to ask the user to specify the time slots when they want to informed about the top news of the day. After all personalized service is the best option.

Turns out, while the notification based on multiple time slots is useful from a personalization angle, it is not very useful from a news “function” angle. A little bit of introspection reveals the “better” formula for notification for a news app that has an archive, a related news technology engine, a summary generation engine and a notification engine – sending the BIG news of the day at the end of the day!

Consumers are caught in their daily grinds and are juggling a barrage of work, personal and social messages. But all consumers want to be aware of their world, the news that matters to them. What better than to send them a capsule of the top news of the day and provide a quick way to catch up on the BIG news with the means to research any specific story in as much depth as possible. After all this is what the 9 pm TV news is all about – top stories of the day; 1 min, 60 stories; headlines of the day are all about giving you the top news as curated by the editorial team in a quick 5-minute recap and then debate the top 3 issues over next 1 hour.

What if we could send an automated summary of all the top 30-50 stories of the day, with some reasonable threshold to determine what is a top story – based on # of related articles, influence of writers, influence of topics / people, # of people reached etc etc.

What if you got a friendly notification from your news app alerting you of the top stories of the day with a quick summary scan and the ability to select any story and check out the depth. Now that seems like an interesting notification / engagement strategy!

So get ready as we get back to the development board and try and hammer this out as quickly as possible. Deadline for the team – 30th October launch of our smart notification – The BIG news of the day.

Comments?

Missing Since August!

For those of you who follow my blog and are wondering why the long silence since my last post of 31st July, my apologies. I have been busy getting all of the ideas covered in this blog into working products. In the last 3 months we have:

* Released our app “inNews” with the smart summary technology, 7-day history, 360-deg media (all related articles, pictures, videos) and forced folks in the news / tech domain to sit up and notice our tech prowress

* Released update of “Filmi Filmy”, our hindi film song app with all songs shown in hindi script, voice search in hindi and an attractive IAP to buy credits for voice search

* Released update of our app “ThotOfU” with ability to pick user’s own image from the device or cloud storage account (Flickr, dropbox, facebook, instagram etc). So now you can design a card using your own memories and bring smiles to your friends.

* Helped release two new service aggregation apps – a wedding services aggregator and a party services aggregator

* Re-position the Pariksha website to reflect our new identity – a leading publisher of Indian content apps and consulting services in app development, web technologies and marteching (marketing + technology). Check out www.parikshalabs.com to see our new identity.

Now that the engineering for the apps is mostly done, there will be time to look at the next set of technology / user experience features to maintain our R&D lead. I continue to believe that in the online / digital space, small but highly useful increments in technology / user experience can create large shifts in user adoption. If you have ideas for our apps, do send them our way.

Escaping The Boring “Happy Birthday” Pattern!

With over 500 friends on Facebook, my school batch on WhatsApp & Fb group; My IIT batch on WhatsApp and other active social contacts, each day I am witness to countless”Happy Birthday” messages posted on Facebook, WhatsApp and other digital forums. Tens and sometimes hundreds of folks post “Congratulations” when someone announces an accomplishment or special day.

Surely, life would be much better for the sender and the recipient if we could post customized, personalized, graphically rich messages instead of the boring, monotonous and repetitive plain vanilla text messages. Surely one is not simply posting out of habit or peer pressure but out of love and concern and engagement with the party, no?

But how does one do this without taking a great deal of time and effort? Some folks do spend the time to scout for interesting cakes or pictures and their messages always seems to stand out and be appreciated. Wont it be great if we could enable everyone to create personalized visual message cards/ greeting cards that could be posted quickly ? Wont it be great if we could bring curious, fun, playful images together with all the images a user has stored on the internet services and let them use these images in their cards?

Here is a small attempt at moving in this direction. ThotofU was launched last year as a “Birthday” greeting app but we realized that it has a wider horizontal use-case, make custom greeting cards for any occasion.

So take out the boredom from your “Happy Birthday” and/or “Congratulations” messages and just use ThotOfU to design beautiful cards that can be shared via Facebook, WhatsApp or even email.Try it and see how other friends “like” your creative expressions.

It takes less than 5 minutes to create and post a card. Simply choose a card template; select & scale a picture, type your message, set size, font, color and placement of the message and your card is ready to be sent! We have a large gallery of pics that can be used for any occasion, whether your Thot is smart, quirky or serious. And we are expanding the gallery all the time.

Sign-up and get 50 credits sign-on bonus. Use the credits to buy pictures from our library for use. When your credits are exhausted, you can buy new credit points and use the points to purchase pictures that you like. Once you purchase a picture, it can be used unlimited times. Very soon, we will let you pick your pictures from any social network and any photo storage service as well.

splash_screen_new-209x300

Updated for the Android Play Store at this time (iOS update coming soon).

Push Notifications for a News Application

What should be the policy for push notifications for a news app? With early each app sending push notifications most users have the ubiqutous “red star” on most apps on their devices. Does sending a notification for every new story make sense? Does sending a summary notification make sense? Should we think of customized, personalized notifications.

Here’s a quote from a friend I think of highly as a good thinker of new tech.

“I have stopped following any press these days. No news channels nor economic times or any such information now. For me most news has become non-actionable. The media content has become exponential both from generated content or syndicated content standpoint. After sifting through pages of content you are still trying to find the actionable items in those. I have started following social media and have started calling some of my connections JENA (Just Enough News Agent). The job of this person is to tweet around what he has read and if you have a few good connections they provide you a pretty descent exposure to the filtered new events. ”

How do we make the notification actionable. What sort of news content is actionable?

In past articles, we have talked about building inNews as an aggregator that determines the BIG news at this time based on the volume and velocity of news articles on that story line and presents the top news with all related articles. It seems logical then to offer a customized, personalized option to the user to choose to receive notifications which will bring to focus the BIG news at that time so the user can check it out. But what will make it actionable? Can we link the notifications to the temporal value of news consumption?

As a user, I may be interested in the top news in the morning to know and also to re-affirm my knowledge of the top news stories from the previous day; by mid-day as I want to catch up on any major breaking stories during the day (especially financial or political) and end of the working day so I have a mental framework for catching up on the news in the evening as needed.

So if we link curiosity about the “news of the day” with the temporal and offer an option to the user to choose which notifications she is interested in, can we bring new value to the user? The answer of course lies in offering the solution to the user and testing the hypothesis.

Coming up in the next inNews release: personalized push notifications bringing you the top news of the morning, mid-day and evening. So that you stay abreast of news. And we get to see you daily. Stay tuned.

 

Week-end Musing: Why Do Indian Music Apps Show Hindi Film Songs in English Text?

Have you wondered why all the great hindi film music on computers, mobile use English (phonetic) to present song titles ? Whether it is YouTube or Gaana or Spotify or any other service, it is common to see English script being used for hindi film songs.

As Gabbar would say, “बहुत बेइंसाफी है”. At Pariksha Labs, we think it is time we fixed this. It is time India adopted Hindi more widely on its computers and mobile. It is time Indian people moved to more natural language on computers and mobiles. Its time we included the non-english speakers, a large majority into the circle. Its time we #DesignedForIndia.

Stay tuned for some big news next week from our music app ‘Filmi Filmy”. क्यों, बात पसंद आई ?

Can Automated Text Summary Technology Work For News ?

 

Well, judge for yourself. The summary below is of a recent new story of a Pakistan trained doctor who has been forced to sell shoes in Ahmedabad. The summary has been produced by our automated engine that we plan to integrate with our News app, “inNews: The Big News Now”. Coming soon.

Summary

Pakistan trained doctor sells shoes in Ahmedabad

He fled to Ahmedabad in 2006 fearing for the safety of his family. Without the MCI’s permission to practice, these doctors often work in pharmacies or even mobile repair shops – all for a pittance. Most of the Pakistani doctors fled to Gujarat for safety. In Ahmedabad, he works at his cousin’s mobile store right next to the airport.

Pak-Doctor

Do share your feedback.