Tag Archives: algorithms

Why Big Data Analytics is harder than you think: numerical precision matters!

How big is your Big Data data set? If it is not measured in petabytes then perhaps it doesn’t qualify. In any case, as the amount of data grows the challenges grow disproportionately and in areas that perhaps you don’t expect. In this post, I am raising a number of challenges and concerns that you should be thinking about, especially if you are interested in business analytics on large data sets.

I recommend that if you are serious about large data sets and numerical precision, you have no choice but to adopt 128-bit arithmetic in most scenarios, especially financial.

Continue reading

C5 Collection Library for .NET

In the last few days I have been reviewing my knowledge of C++ and the STL (standard template library) in particular.  I have often found it frustrating that the standard .NET collection classes omit several of those in the STL, most notably a simple and efficient set implementation.  Using Dictionary<T, bool> and TryAdd is just not elegant!

I have just found the C5 library from the IT University of Copenhagen with full documentation and academic analysis.  Do you use it?  What are your experiences?