Usually we don't write much about the underlying technology that makes Big Data Lens work. We decided to change that here in 2014. We doing it for several reasons.
One is the contribution sharing experiences about the use of different technology makes to the tech community as a whole. We benefit from others experience and so we should make similar contributions to others. its just part of the tech ethos.
Another reason is we just get excited about certain technology when it works well. The Go Language is a case in point. Now we know not to many readers get excited about about a programming language so we'll keep it high level.
First, the Go Language http://golang.org/ was started by Google. We don't know for sure but assume some of Google is built on this language. So it must be good. Turns out it is. As the website states;
"Go is expressive, concise, clean, and efficient. Its concurrency mechanisms make it easy to write programs that get the most out of multicore and networked machines, while its novel type system enables flexible and modular program construction. Go compiles quickly to machine code yet has the convenience of garbage collection and the power of run-time reflection. It's a fast, statically typed, compiled language that feels like a dynamically typed, interpreted language."
So that is a mouthful but what it really means is the this language is FAST. We have written our core Natural Language Processing (NLP) in Java, Kotlin and now Go. Go gives us about a 30% speed advantage over the other compiled languages. We no longer measure our processing in milliseconds (a thousandth of a second) - we measure only in microseconds (a millionth of a second).
Fast processing is the key to making Web Services work on a economic and customer service level. A 30% time savings on a small batch of processing is nice. But a 30% savings on Big Data scale processing is huge and means less cloud costs, less time customers wait for results, and savings that can be passed on in lower prices.
We'll detail other aspects of the Go language over the course of the year. But there is one more aspect worth mentioning - the icon. Every really good programming language has a some icon to it up. Used to be very techy kinds of things to show the seriousness of the effort. In later years we have switched to friendly looking animals or robots. Go has this happy looking gofer ready to chew through the toughest problems.