Geek Out

Komli Media’s goal is to be one of the leading digital technology and advertising companies in Asia.Komli Media is driving a revolution in digital advertising, being one of the few companies in the world to provide real-time display, mobile, video, social, and search solutions for advertisers, agencies, and publishers, while leading the charge in the next generation of digital advertising technologies. By using cutting-edge technologies (e.g., Hadoop, NoSQL, netty), at scale (systems to process hundreds of thousands of request per second), and with big data (terabytes of data per day), Komli Media aims to revolutionize online advertising and how it’s delivered.

Revolution in Display (display, mobile, video)

Some background
Fifteen years ago, Yahoo! ruled search. It organized all search results manually into a catalog and talked about “curated results”. This led to mediocre search experience for consumers and advertisers, but this was all the world knew. Then Google came along, and brought algorithms, data, optimization, and targeting to search and transformed it! Search became so much better for the user and so much more targeted for the advertiser. Google gets approximately a 2% CTR on search (click-thru rate = for every search a user conducts what percent of the time does the user click on an ad). This became one of the most effective forms of advertising ever known and transformed search from a $200M advertising market to $25B advertising market over the last 12 years.

Current day
Display advertising is as big a market as search advertising. Display, like search in its early years, has also been primarily manual in that ads are served based on the site the user goes to. To oversimplify, display today gets a 0.1% CTR. But in the last few years the technologies available to display have started changing dramatically and display effectiveness is starting to close the gap with search. Now display advertising is set to grow faster than search and be bigger than search advertising!The same technologies that Google brought to search are now being brought to display by Komli Media. Komli Media’s goal is to make display advertising as effective as search advertising – effectively “bringing the science of search to display.” How are we doing that? By bringing together sophisticated algorithms, the power of analytics and data, and real-time technologies into a cutting-edge system that revolutionizes how the display ad world works. Komli Media is developing and leveraging technology paradigms such as:

  • • Real-time bidding (RTB) – Imagine that in the time you go to a webpage or a mobile site and an ad is going to show to you, hundreds of buyers bid on that ad in real-time and determine in real-time the price they are willing to pay for that ad. Similar to a stock exchange.
  • • Web scale – How do you build systems to handle billions of requests per day, how do you build a system to handle hundreds of thousands of requests per second? Are we scaling up, or out, or both? And can you do this cost effectively, in an automated fashion? By the way, are you ready to handle anything the Internet or the world, frankly, throws at it? Boxes go down, no problem. Data centers go down, no problem. Underwater data cables go down, no problem. Traffic seamlessly moves somewhere else in the cloud and customers keep getting their job done.
  • • Big data – Komli collects many gigabytes of data a day, and that will soon go to terabytes of data per day. This data is about billions of ads served, clicks, impressions, data events, consumer behaviors – how do you process all this data and derive insights to improve our algorithms or highlight potential improvements to customers?
  • • Ad price prediction – For an ad you are going to buy or sell, how do you decide how much that ad is worth, and we only have a few milliseconds to make a decision? Is this the user who is going to buy a ticket from MakeMyTrip, or is this the user who needs a test drive for a new Fiat car, or is this the user who is thinking about switching to Pepsi from Coca-Cola?
  • • Data market places – Did you know that if you run a search on a travel site you might be marked (anonymously) as someone traveling from Bangalore to Mumbai next week and then later shown an ad on the internet that gives you specific prices for a Jet Airways flight from Bangalore to Mumbai.
  • • Click fraud – If you have millions of users seeing your ads on thousands of websites, how do you determine if any of those users or sites is conducing fraudulent actions?

And many more that we cannot list here. All of the above require using open source technologies such as Hadoop, Flume, Oozie, Cassandra, Memcached, Redis, Zookeeper, Capistrano, Thrift, etc., as well as proprietary platforms on a day-to-day basis.

Revolution in AUDIENCE

Advertisers are increasingly looking to target users who are the actual consumers they want to sell their product to. It doesn’t do very much good for Porsche to target their marketing to a child in 10th grade. For years markets have been looking at ways to get more and more targeted, but at the same time to reach those consumers at scale. For example, Porsche wants to reach five million wealthy males at one time, rather than two people at a time. Porsche need to reach a lot of people in their target group at one time, because it’s much more efficient.

Doing this kind of targeting in television or print has never been possible, and people have long talked about 1 to 1 marketing in online, but the technology wasn’t there to really make it happen. Komli Media is making it happen in online advertising, now! Komli wants to help marketers reach millions of the RIGHT consumers across the internet, at the right time. But how do we do that? Easy. Well, not really. By watching the activities of hundreds of millions of consumers in terms of the ads they click on, the actions they perform online and offline, the time they spend , the purchases they make, the websites they view, Komli looks to predict the behaviors and attributes of consumers and then serve them targeted advertising.

This requires sophisticated data mining techniques to sift through terabytes of data on a daily basis and predict who users are, and what they are going to do next. as:

Pushing the Envelope in Technology

RTB (real-time bidding) gives our advertisers their perfect audiences at market prices. Knowing or inferring about our audience better than competing bidders and placing a bid in real-time based on what we know about each individual visitor is the key. This is partially a data management problem on a massive scale with tyrannical requirements. Here is just a teaser of some of the challenging problems in building such a platform:

  • Aggregation: Data of various types: click stream, behavioral, purchase and campaign data is received from a variety of sources such as search, social, site and mobile. Most of this received data will be massively sized and each entry speaks about a particular point in the purchase funnel. Which algorithms and data structures are best suited to aggregate, represent and summarize this data?
  • Correlation: What analytical model shall we use to form meaningful correlations and which heuristics or machine learning techniques are best suited to summarize data at this scale? Which techniques will allow us to have a fast turnaround time in improving the algorithm? Can we allow multiple RTB systems to use the correlated data? If so, what mechanisms will we use to keep specific rules from interfering with each other’s learnings?
  • Freshness: Certain intent data expires quickly – e.g. an individual with the “travel” tag tends to be far advanced in the purchase funnel right from the time we first see them tagged so. To send this data through expedited processing may or not may mean the difference between getting a steal on a “travel ad” and withholding bidding because the tag expired. What algorithms or simplified models can be applied in each case? Does it make sense to promote a tag as a “fast tag” automatically?
  • Real-time access: Which data structures / data-stores are best suited to organize this data? How do we ensure that this system is fast for point-accesses of particular user data as well as custom query on ad hoc features (e.g. automatic age-groupings) of the audience. How does this interplay with (2) and (3) above?
  • Simulation: Can advertisers do a simulation or shootout of various audience targeting techniques by simulating the effectiveness of different ad campaigns / budget allocations using this platform without actually spending any money? How much analysis can we help them do in this manner?

Of course, features like accounting, auditing, data visibility through dashboards, self-serve interfaces etc., and concerns like application of business rules, compliance with NAI, monitoring, multivariate testing, reliability in numbers, archiving etc., are ever-present in this mix of problems to make life, well, much more interesting!

How would you like to be a part of piecing together some of the tech aspects of this giant puzzle? For each of these problems you will have an opportunity to work with an open mind to imagine what is the best possible solution to the problem and how much can be obtained by leveraging existing technologies. Often a performance benchmarking showdown between competing technologies may be warranted. Here’s a flavor of some of the current technologies being used/considered:

  • HDFS with map-red jobs alone or a combination of those with HBase.
  • Thrift, Avro, ProtoBuf.
  • Kyoto Tycoon v/s a proprietary NoSQL DB.
  • Taxonomy graph, trees, sparse matrices, N-dim matrices.

At Komli, you will work with some of the best minds in the tech industry and build innovative media and advertising platforms.

Summary

As you can see, the opportunities in display are tremendous. Komli Media is focused on building the next generation platform across digital, allowing marketers and publishers to profit in ways never seen before. This requires cutting-edge development to build at web scale with huge datasets – the web will take nothing less.