Author Archives: Steve Horsfield

Indispensable Linux sysadmin tools for the developer (in a DevOps world)

For many, development and system administration are alien worlds. Developers understand Java, .Net, Python, Ruby, Elixir, Go, Rust and many more, but ask them about IPv4, ARP, DHCP, systemd, /etc, swap, awk or sed they go blank. These are not things discussed for many developers, especially those used to Windows or Mac development where much of this is hidden or little used.

Enter DevOps. With DevOps, developers can no longer say that how the system operates is not their concern. As a team, they need to build each other to have at least passing understanding of the vagaries of system administration.

But not only is it about sharing knowledge and ownership, it’s also about being competent, indeed useful, in another context. Seeing opportunities where system administration skills can enhance the quality of the product now that there is no longer an artificial divide between these two worlds.

The following is a short list of extremely useful tools to have available in your tool belt for system administration. I strongly recommend that you develop a passable understanding of these. You’ll be glad you did.

Continue reading

Advertisements

Book review: The Go Programming Language, Donovan & Kernighan, 2016

This is a fairly good book to read, especially if you are new to the Go programming language. It also serves as a useful reference while you are getting familiar with the Go language. However, it is not a serious contender for a programming language reference guide such as The C++ Programming Language, by Bjarne Stroustrup. That also makes it somewhat easier to read!

Background

I have extensive experience in many programming languages (C, C++, x86 Assembler, BASIC, Java, C#, F#, Visual Basic) and passing familiarity in several (Lisp, Pascal, Fortran, OCaml, EcmaScript)━I’ve probably forgotten some━but I’m a relative newcomer to the Go programming language and ecosystem.

In the course of the past few weeks I have started my journey in Go, writing fairly substantial utility programs and providing one (so far) small pull request on a major and well-known open source product, Hashicorp’s Terraform. I have also read this book cover to cover, as it is a learning style that has worked very well for me in the past.

Who should read this book?

This is a book for developers, but you should already be proficient with at least one other mainstream imperative programming language (C, C++, Java, C#, etc.) before tackling this book because it does presume a fair amount of basic programming competence.

Obviously, you should be interested in the Go programming language, but ideally you should have opportunity to use it on some real implementation work, ideally something more substantial than the exercises in the book. Those exercises will almost certainly help cement an understanding of the material in the book, but they are not a substitute for real programming challenges, particularly as they guide the reader in the overall design and limit the scope of thinking━exactly what you need in exercises, I think. I didn’t do any of the exercises in the book, that’s just not how I roll.

If you are already proficient in Go, this book isn’t likely to tell you much that’s new━perhaps one or two things about the dangers of code involving uintptr and unsafe.Pointer, but not a lot else. It certainly isn’t going to stop you having to read package documentation, even for standard packages like fmt.

The bad

I’ll present the bad things first so that by the time you’ve finished reading this post━if you make it that far!━you’ll be more likely to remember the good parts.

  • This book won’t take you far into mastery of the language. It will help you become competent, but won’t really get much further than that
  • The authors━or at least one of them━provide some very spurious arguments to justify particular language design elements. For me, these were irritating but thankfully they became less frequent after you passed the halfway mark
  • The early chapters are particularly slow. It’s quite easy to get frustrated at the pace, but if you haven’t promised to provide a review you might find it easier to just skim read those parts, but don’t think you are buying a book with 366 pages of dense language instruction, I think they could have covered the same material in half the number of pages, particularly if they removed or shortened some of the examples

The good

  • This is a good book. I would not call it a great book, but it is a good book
  • The authors appeal to several learning styles: decent textual coverage of content, example code (available to download and play with), guided exercises of varying complexity
  • There is extensive coverage of each language element. Go is a fairly small language with few edge cases, and so there is a decent amount of content for each and every construct in the language
  • The main toolchain (go run, go build, go get, go test, go install, go doc) is introduced, including brief coverage of the way imports are handled, how to develop automated tests and benchmarks, review test coverage and, of course, create executables
  • Decent introductory coverage of how to write concurrent programs in Go (two chapters), including certain design approaches, e.g. event broadcast using close
  • Basic introductory coverage of the unsafe and reflect packages, including a simple example of interfacing with external, C library code

Where to go next

Since the book doesn’t really get to mastery of the language, where do you go for more details?

Book details

  • Alan A. A. Donovan · Brian W. Kernighan
  • Published Oct 26, 2015 in paperback and Nov 20 in e-book
  • Addison-Wesley; 380pp; ISBN: 978-0134190440
  • http://www.gopl.io/

Simple coloured output from CMD.EXE

It’s stupid to have to post this, but it’s even more stupid that the function isn’t built in.

Suggestions on the glorious Internet are just nasty: use PowerShell or use hacks around other built-in functions.

Instead, I’ve thrown together a 10KB C++ console app that starts and exits almost instantly.

To use it, you simply pipe what you want to display to the executable. You pass in one or two heaxdecimal digits as a parameter and that colours the output accordingly:

screenshot_colourecho

You can use the colour values in the wincon.h header file. These are fairly limited, so you may be best off trying the various values. The first value is the background and the second is the foreground. Not all values are supported for the background.

 

The code and executable are available here:

https://app.box.com/s/nnyv082efsiedhl2rrxw0qs761jm9t1w

 

 

Why Big Data Analytics is harder than you think: numerical precision matters!

How big is your Big Data data set? If it is not measured in petabytes then perhaps it doesn’t qualify. In any case, as the amount of data grows the challenges grow disproportionately and in areas that perhaps you don’t expect. In this post, I am raising a number of challenges and concerns that you should be thinking about, especially if you are interested in business analytics on large data sets.

I recommend that if you are serious about large data sets and numerical precision, you have no choice but to adopt 128-bit arithmetic in most scenarios, especially financial.

Continue reading

Performance in SQL Server 2012 degrades non-linearly as SELECT clauses are added when using ColumnStore

When adding additional SELECT clauses to an aggregating query over a large fact table and using ColumnStore, the performance degrades in a step-wise linear fashion with large steps.  It may be quicker to execute several less-complex queries rather than a single complex query.

I’ve submitted a Microsoft Connect bug report here: https://connect.microsoft.com/SQLServer/feedback/details/761895/.

Continue reading for full details, or download this attachment: https://www.box.com/s/5hp9f0ditg0fspghu506 [PDF file also available via Microsoft Connect].  Actual test results, steps to reproduce, etc., as well as more pretty graphs :), are included.

Elapsed time by SUM measure count (base query)

Continue reading

Performance in SQL Server 2012 degrades when using ColumnStore and no GROUP BY clause

Essentially, the performance of a non-grouping SQL SELECT query degrades when applied to a ColumnStore index.  This has been tested with SQL Server 2012 RTM CU1.  Performing partial aggregation can result in a 15x performance improvement in some of the cases I observed.

See the full details in my Microsoft Connect submission here: https://connect.microsoft.com/SQLServer/feedback/details/761469/. Or download the details here: https://www.box.com/s/frf7imhyclb2efz2tfvb.

Continue reading