Category Archives: Information Technology and Systems

Book review: The Go Programming Language, Donovan & Kernighan, 2016

This is a fairly good book to read, especially if you are new to the Go programming language. It also serves as a useful reference while you are getting familiar with the Go language. However, it is not a serious contender for a programming language reference guide such as The C++ Programming Language, by Bjarne Stroustrup. That also makes it somewhat easier to read!


I have extensive experience in many programming languages (C, C++, x86 Assembler, BASIC, Java, C#, F#, Visual Basic) and passing familiarity in several (Lisp, Pascal, Fortran, OCaml, EcmaScript)━I’ve probably forgotten some━but I’m a relative newcomer to the Go programming language and ecosystem.

In the course of the past few weeks I have started my journey in Go, writing fairly substantial utility programs and providing one (so far) small pull request on a major and well-known open source product, Hashicorp’s Terraform. I have also read this book cover to cover, as it is a learning style that has worked very well for me in the past.

Who should read this book?

This is a book for developers, but you should already be proficient with at least one other mainstream imperative programming language (C, C++, Java, C#, etc.) before tackling this book because it does presume a fair amount of basic programming competence.

Obviously, you should be interested in the Go programming language, but ideally you should have opportunity to use it on some real implementation work, ideally something more substantial than the exercises in the book. Those exercises will almost certainly help cement an understanding of the material in the book, but they are not a substitute for real programming challenges, particularly as they guide the reader in the overall design and limit the scope of thinking━exactly what you need in exercises, I think. I didn’t do any of the exercises in the book, that’s just not how I roll.

If you are already proficient in Go, this book isn’t likely to tell you much that’s new━perhaps one or two things about the dangers of code involving uintptr and unsafe.Pointer, but not a lot else. It certainly isn’t going to stop you having to read package documentation, even for standard packages like fmt.

The bad

I’ll present the bad things first so that by the time you’ve finished reading this post━if you make it that far!━you’ll be more likely to remember the good parts.

  • This book won’t take you far into mastery of the language. It will help you become competent, but won’t really get much further than that
  • The authors━or at least one of them━provide some very spurious arguments to justify particular language design elements. For me, these were irritating but thankfully they became less frequent after you passed the halfway mark
  • The early chapters are particularly slow. It’s quite easy to get frustrated at the pace, but if you haven’t promised to provide a review you might find it easier to just skim read those parts, but don’t think you are buying a book with 366 pages of dense language instruction, I think they could have covered the same material in half the number of pages, particularly if they removed or shortened some of the examples

The good

  • This is a good book. I would not call it a great book, but it is a good book
  • The authors appeal to several learning styles: decent textual coverage of content, example code (available to download and play with), guided exercises of varying complexity
  • There is extensive coverage of each language element. Go is a fairly small language with few edge cases, and so there is a decent amount of content for each and every construct in the language
  • The main toolchain (go run, go build, go get, go test, go install, go doc) is introduced, including brief coverage of the way imports are handled, how to develop automated tests and benchmarks, review test coverage and, of course, create executables
  • Decent introductory coverage of how to write concurrent programs in Go (two chapters), including certain design approaches, e.g. event broadcast using close
  • Basic introductory coverage of the unsafe and reflect packages, including a simple example of interfacing with external, C library code

Where to go next

Since the book doesn’t really get to mastery of the language, where do you go for more details?

Book details

  • Alan A. A. Donovan · Brian W. Kernighan
  • Published Oct 26, 2015 in paperback and Nov 20 in e-book
  • Addison-Wesley; 380pp; ISBN: 978-0134190440

Why Big Data Analytics is harder than you think: numerical precision matters!

How big is your Big Data data set? If it is not measured in petabytes then perhaps it doesn’t qualify. In any case, as the amount of data grows the challenges grow disproportionately and in areas that perhaps you don’t expect. In this post, I am raising a number of challenges and concerns that you should be thinking about, especially if you are interested in business analytics on large data sets.

I recommend that if you are serious about large data sets and numerical precision, you have no choice but to adopt 128-bit arithmetic in most scenarios, especially financial.

Continue reading

Performance in SQL Server 2012 degrades non-linearly as SELECT clauses are added when using ColumnStore

When adding additional SELECT clauses to an aggregating query over a large fact table and using ColumnStore, the performance degrades in a step-wise linear fashion with large steps.  It may be quicker to execute several less-complex queries rather than a single complex query.

I’ve submitted a Microsoft Connect bug report here:

Continue reading for full details, or download this attachment: [PDF file also available via Microsoft Connect].  Actual test results, steps to reproduce, etc., as well as more pretty graphs :), are included.

Elapsed time by SUM measure count (base query)

Continue reading

Performance in SQL Server 2012 degrades when using ColumnStore and no GROUP BY clause

Essentially, the performance of a non-grouping SQL SELECT query degrades when applied to a ColumnStore index.  This has been tested with SQL Server 2012 RTM CU1.  Performing partial aggregation can result in a 15x performance improvement in some of the cases I observed.

See the full details in my Microsoft Connect submission here: Or download the details here:

Continue reading

Why software developers need to look beyond frameworks and languages

My career as a software developer has been fairly unusual as I spent about six years as an IT manager for two companies. It was not an intentional change of focus but happened unexpectedly after taking a job as an Oracle database developer.  That job changed shape immediately, in the form of a job offer for a job I was not interviewed for. It was a big undertaking for both my employer and for myself, but in the process it exposed me to many challenges that most developers have no experience of.

It is my belief that my current software development skills have been enriched as a result. I also believe that other software developers would be able to act more effectively with a little investigation around the edges of their own areas of expertise. In this brief article, I hope to encourage you, my readers, to swim a little further out to sea. I hope you find it useful and can make your way back against the tide!

Testing for correctness

Have you heard of integration testing? This is where you test how the software that you and your team have developed interacts with the surrounding eco-system. Perhaps you already do a lot of integration testing. If not, I certainly encourage you to do it, at least in some form. However, if you are performing integration testing without a consideration of the wider eco-system— networks, storage devices, processing platform, operating system options and configuration, drivers, patch level, etc.—then you are leaving open a multitude of untested areas in your product. Whether or not these risks are considered acceptable or not to your business is a matter of judgement, but wouldn’t it be nice to at least smoke test these? Or alternatively, wouldn’t it be nice to know and understand how the test environment is constrained similarly and differently from the real world environment? Not being able to answer these questions means that you won’t be able to tell the IT manager engaged in deploying your software the answers to his justifiable and reasonable questions.

Product ownership and leadership

One of the major failings in the modern world (I have seen it in governments, commercial businesses, charities, churches and social groups) is abdication of ownership.  More generally, I see this as a failing in leadership, but that is a discussion that is probably best conducted in another post.  A failure in ownership means, typically, that important things are left undone, because there is no one that considers it to be their job to ensure that it is done.  Note how I have phrased the last sentence, “to ensure that it is done”.  It is not always the case that the person taking ownership of an activity is the same person responsible for performing the activity.  Nevertheless, not having someone who is willing, able and empowered to take ownership of the activity defeats success almost every time.  We need more people to take ownership of the things for which they have been deemed responsible, or at least been given responsibility for.

In relation to software development, ownership leads to a need for a product team to understand the eco-system in which their product will be deployed.  I mean, to really understand.  I mean to understand to the degree necessary to be able to describe the entirety of the product’s function for which they are responsible.

This means that they should know how to deploy it.  They should know how it has been tested.  They should know how it has been designed.  Importantly, they must, they absolutely must, be able to describe how to use it, in all of its uses, to their user community.  Finally, they must be able to maintain it and support it.

I cannot see that any of this is possible without the team taking a high level of ownership of the entire product for which they are responsible.

Owning a deployment platform

If this level of ownership is to be achieved, then software developers must aim for at least level 3 of the Capability Maturity Model: Defined—a standard business process.  In other words, they must be able to correctly deliver an equivalent configuration based on a repeatable and well-described process.  This is not possible without understanding the configuration of the operating system, and equally of related functional areas.

For an example, a very simple but typical two tier web application deployed on self-hosted hardware would need:

  • Two physical machines
    • One, single disk system with minimal RAM, CPU and memory
    • One, RAID-1 operating system and database log disk; one, RAID-5 array for all other data files
  • One network switch
  • Two operating system installations
  • One web server installation
  • One database server installation
  • One database schema deployment
  • One web application deployment
  • Two firewall configurations
  • Two security configurations
  • etc.

The complexity quickly builds up.  Is there a SAN?  Is there a central network authentication system?  What protocols are being used? UDP? TCP? IPsec? LDAP? Kerberos? SOAP? XMLA?

Taking ownership of a product means taking ownership of the aspects of this that matter to the product’s correct function.  At the very least, it means knowing how to deploy the software and how to configure each part of the overall solution for correct behaviour.  It does not mean understanding everything; it just means understanding enough.

We need software developers to recognise that they develop products deployed into eco-systems, not products acting in isolation (well, at least that is true for most of us).  Therefore, it is critically important that we take responsibility and ownership for what we know and for what we require to be controlled.  We need to improve the quality of software development in the industry.  Would it not help if we at least could describe how the software we were responsible for creating works?

Chrome almost supports SSO in Windows Kerberos environments

I was pleasantly surprised to find that Google Chrome has support for SSO and the Negotiate algorithm. Indeed it also has support for NTLM. So why the need for this post? I think the implementation could do with a little refinement.

Here’s my assumption. Credential delegation in a Kerberos environment is managed by the Kerberos system and its configuration, clients should not attempt to interfere with it. However, Google Chrome disallows ticket forwarding by default, effectively preventing delegation (constrained or otherwise). You can change this with an option on the command line but that means you have to know the option exists and have to plan to change it for every user of your web site. Seems the wrong way round to me. This default means that, out of the box, most web sites of any complexity will not operate as per their intended design.

Secondly, the default SPN behaviour is incorrect for Windows platforms. The Kerberos specification does not say much about SPNs, but they do at least have several parts: the service type, the host and port, and optionally an additional service identifier. Including the port is standard, but Chrome doesn’t do this by default. Secondly, the Chrome default behaviour is to resolve DNS CNAME records to A records and use this for the host part. I can’t fault Google for this approach but it does differ from the widely documented Windows approach of using SPNs for the host header (i.e. before CNAME resolution). (As an aside, note that if you take that approach then why shouldn’t you use the IPv4 address, or the IPv6 address, and what if the machine is multi-homed?). It also interferes with the ability of a host to provide multiple independent services because with the Google approach they all have identical SPNs. In Chrome’s defense, these options can also be controlled via the command line.

Finally, note that NTLMv2 is only available on Windows platforms. Chrome supports NTLMv1 on other platforms but that is horrendously insecure! This is not intended as a negative comment on Chrome, just something to be aware of.

It is great to see other browsers finally supporting SSO, Negotiate, NTLM and Kerberos. I just hope that interoperability is considered a desirable end goal. Without it these are just more competing proprietary solutions, and that would be a shame.

Material about Google Chrome was taken from here: HTTP authentication [The Chromium Projects]. See my recent post about Kerberos in Windows for links to supporting Windows implementation materials.