Category Archives: Programming

Provided Scope in Gradle

A few days ago, in Android Dependency Double Trouble, I needed to add something like Maven’s provided scope to a Gradle Java project. I learned how to do it from How do I best define dependencies as “provided”?, which offered two methods. The first looked like the simplest, but I couldn’t get it to work. Still, I got the second way working, and then moved on. I left myself a note to come back someday and figure out how I was messing up the simpler method.

I’d used the provided scope for a dependency from a project named TestSupport onto one named Core. Today I hit a problem when adding tests to my TestSupport project (that’s right, tests that test the code that supports my tests): the tests couldn’t find classes in the Core project. While flailing around to try and fix it, I learned more about the differences between the two methods of dealing with the provided configuration, and how to have fully working builds, with TestSupport tests, using either method. Continue reading Provided Scope in Gradle

Commutative equality in Ruby?

Today I discovered Ruby doing something unexpected with the == operator. With a class like this:

class Value
  # ... initialize, other methods, etc. ...
  def ==(other)
    self.a_method == other.a_method

and a test that effectively evaluated the expression:

  3 ==

I expected a message saying that Value couldn’t be coerced to Fixnum. Instead I got

undefined method `a_method' for 3:Fixnum (NoMethodError)

I thought that 3 == was equivalent to 3.==(, yet this message suggests Ruby is evaluating It’s almost as if Ruby has decided that == is commutative. How can that happen? I was obviously missing something. Continue reading Commutative equality in Ruby?

Fiddling with Functors

Learn you a Haskell continues to bend my mind. This week we were looking at the first half of the chapter on Functors, Applicative Functors and Monoids. I feel I’m following along with the text okay, but just getting brief glimpses at the strange beasts that that lay beneath. While noodling around with one of the examples

ghci> (:) <$> Just 3 <*> Just [4]  
Just [3,4]

I wondered if the type system could infer one of those Justs. Continue reading Fiddling with Functors

Point-free baby-steps in Haskell

we7‘s study group (still aka Nerd Club) has moved onto a bit of Haskell with Learn You a Haskell for Great Good. The chapter on higher order functions introduces this definition:

numLongChains :: Int
numLongChains = length (filter isLong (map chain [1..100]))
    where isLong xs = length xs > 15

For some reason I immediately wanted to factor the thing as a function taking isLong as a parameter: Continue reading Point-free baby-steps in Haskell

Hive: multi-insert and parallel execution problem

I’ve been having trouble with Hive after I added a SELECT clause to a multi-insert and started seeing java.lang.InterruptedExceptions. What follows is the smallest example I’ve been able to put together to demonstrate
the problem. It fails most of the time, but will just occasionally run without problem.

The query is:

  SELECT a, b
    FROM input_a
    JOIN input_b ON input_a.key = input_b.key

and the error from the hive CLI client is: Continue reading Hive: multi-insert and parallel execution problem

Hive UDFs in views

You can create user-defined functions in Hive. Simple ones are simple. The syntax for declaring a function is also simple:

CREATE TEMPORARY FUNCTION my_func AS 'in.sinking.udf.MyFunction';

What’s that TEMPORARY doing there? Well, it means that my_func is only available during the current hive session.

I found myself creating a VIEW that uses my_func – how does that work? Pretty well, as long as you only query it from the same hive session in which you declare the function. When you next fire up hive you’ll find your VIEW mysteriously fails with:

SemanticException Line 16:4 Invalid function '`my_func`' in definition of VIEW ...

Gah – that took me a while to figure out. The workaround seems to be to bung the CREATE TEMPORARY FUNCTION ... clause into your .hiverc, thereby making it a bit more permanent. There seems to be an old issue on a related subject in this issue on Hive’s Jira.

Seven Databases: MongoDB and Cities

A few weeks ago the “nerd club” reading group at we7 moved on to Seven Databases in Seven Weeks. It’s a fun book, and I’ve been enjoying working through the exercises. The chapter on MongoDB has an exercise to use the geospatial indexing feature to search for “cities” near London. After a bit of digging and some pretty pictures I discover that things are not quite right with the supplied data.

Continue reading Seven Databases: MongoDB and Cities