Thoughts on Shen

What is Shen?

• Practical version

Shen is an hosted Lisp that comes with a full-featured macro system, a Prolog, an optional type system more powerful than Haskell’s, and does it all in under 5000 lines of code. It has been ported to many platforms including Java, JavaScript and Ruby.

• Emotional version

It is the magnum opus of a lone programmer who is well-versed in theory AND practice. Even if you don’t think the language might be worth more attention outside of this talk, it might give you a few cool insights.

The essay linked under `practice` is one I feel so strongly about that I must restrain myself not to tell it over and over to every new coworker. Guy tells a bit about my personal story - only I didn’t return to academia to see what I could gain by studying more.

Why Lisp?

Most people could rephrase it as ‘these parens hurt my eyes!’. Rest assured that they matter. We Lispers understand that they’re there for a very important reason, and that reason is:

Lisp is the programmable programming language.

Btw, there are tools to alleviate most problems with parens. There’s the awesome Paredit, and the subject of M-Expressions always comes up (we only need more developers to actively work on this!)

Problems in paradise?

As told in “The bipolar Lisp programmer”, Lisp loses culturally because it’s so easy to do complicated stuff in it that no one cares to document and package nicely for “lesser” mortals.

(Ironically enough, I’m writing this while in one of my own insomniac intellectual frenzy of productivity. Especially in my country - Brazil - I find a teacher who understands the finer psychological aspects of learning and productivity a very rare sight. Perhaps we can learn from other cultures how to do university better?)

There’s a lot of historical baggage and lingo that is very off-putting for new developers (“head is car and tail is cdr? What do these even mean?”)

The very nature of the language gives a lot of space for the programmer to make mistakes. It is powerful, but should be wielded wisely.

(As with most dynamic languages, in fact.)

One coworker wrote a very cool piece on some of the problems he found in the (Common) Lisp language. I agree with the sentiment (I have spent some time with Haskell because of this), but I feel that Shen is a step ahead, by joining the best of both worlds.

Got it, Lisp is cool but not perfect. How does Shen compare?

Shen has a Haskell-style type system that is entirely optional. This allows one to opt-in or out of this feature whenever needed - so you get the best of both worlds: the flexibility of dynamic languages and the feedback on compiling mistakes of “static” languages.

It respects and improves on the venerable Lisp tradition of easily creating DSLs by providing a built-in, baked-inside-the-language YACC. This, coupled with the fact that Shen is homoiconic, gives you a lot of power with very low operational overhead. It is used to implement the built-in Prolog in what I consider a very small LOC for the amount of functionality it provides.

(Yeah, there’s a defcc “primitive”. How cool is that? Perl 5 people should be envious.)

(To clarify: what is “operational overhead”? I want to say that it’s easier to do JetBrains-style stuff when the language allows you to. Better than having to muck around parsing code, and deal with the problems related to it. No syntax = one less problem to focus on.)

Strangely enough, it is a Lisp with the Unix-virusesque quality (as argued in the body of essays collectively named as Worse is Better). It compiles down to a mini-Lisp called KLambda that is very easy to port to different environments, and this has been done many, many times.

Shen sounds very cool. Now, I’m sure it’s not perfect. Can you tell me about its problems?

Sure! There are a few:

The community is definitely niche. At the time of this writing, the mailing list for the project is around 500 people - IMHO, it looks like it’s mostly academics who bend towards the practical and like to “get their hands dirty” on code. Almost the same demographic one would expect to find in Haskell mailing lists.

The language’s author is known for being a bit harsh towards open source. It difficults adoption (no self-respecting developer wants to be forced to use something [s]he can not read the source and tweak, if the need ever arises). The author argues eloquently about this to some extent here, but, as one of the so-called “Twitter generation” myself, I have to say that I only started taking Shen more seriously after their website was revamped.

Wait, what do you mean? There’s no open source?

Yes, there is.

You can understand the author’s stance on open source by looking at it through the point of view of the Lisp community. There is a recurring theme of pride in using what are called professional-level tools. The community understands that such a degree of power can only be used by people who committed themselves to understanding how to use it - and the level of dedication required for that usually means “for my entire lifetime”.

Given that the Lisp arts are alien to most programmers, the level of investment necessary to use them “in the real world” is much bigger than less “weird” counterparts. So, paradoxically, Lisp respects its users’ intellectual ability, but exactly because of that, it demands more of them, while other languages might give a “quick fix” by appealing to what is easily relatable (as a famous example, JavaScript having C-style syntax, opposed to Scheme syntax as Brendan Eich originally envisioned).

And so, it remains (and probably will remain) a niche language. It gives you very, very much, but it “only” requires that you dedicate yourself completely to it. Ever read the piece about Haskell being “that girl, the special one”? It’s kinda like that.

Obligatory xkcd on this matter. It tells more than I can put into words, really. Also, I need it to finish this.

Takeaways

Shen is a cool language in the spirit of Lisp that modernizes it and is backed by cutting-edge research on programming languages. It gives very thoughtful answers to problems we, the programmers’ global tribe, are collectively facing.

The community is small, but well-meaning and full of smart people. Exactly the sort of people you want covering your back when you’re facing a difficult bug.

We still need to solve the “Lisp paradox”, but we’ll get there, one step at a time.

Resist the seductive argument of using “real world” languages and choose one that has the best of both worlds. We are the rebels. Join us!

Linking to Github directly from Emacs

GitHub has a RESTful interface for showing files in different branches and commits. In my daily tasks, it’s common to refer to files in different branches and commits for code review, so I knew I had to make this more efficient. Thus the following code was born:

(defun github/copy-file-url (curbranch)
  (interactive (list (magit-read-branch "Branch: ")))
  (let* ((toplevel (replace-regexp-in-string "\/$" "" (magit-toplevel)))
         (curbranch (or curbranch (magit-get-current-branch)))
         (pathtofile (replace-regexp-in-string (regexp-quote toplevel) "" (buffer-file-name))))
    (message
     ;; format: $REMOTE-URL/blob/$BRANCH/$PATHTOFILE
     (kill-new (format "%s/blob/%s%s#%s"
                       (replace-regexp-in-string "\.git$" "" (magit-get "remote.origin.url"))
                       curbranch
                       pathtofile
                       (mapconcat (lambda (pos) (format "L%s" (line-number-at-pos pos)))
                                  (if (region-active-p)
                                      (list (region-beginning) (region-end))
                                    (list (point))) "-"))))))

In the top let* expression, we get the current branch and the path to the current file relative to the repository’s toplevel. I usually clone via HTTP, so remote.origin.url returns an almost correct URL (we need to strip it of the trailing ‘.git’ to work). Finally, we get the line number(s - plural if there’s an active region) from Emacs and build the Github URL.

I didn’t test it, but would be happy to discover that this function also works with git-timemachine.

Contributing to Spacemacs

Recently I touched a Rails/JavaScript codebase and I needed to tweak my Spacemacs a little. The defaults are great, but I felt that some things were missing: better integration with Rubocop and a REPL for JS. So, instead of making it a private layer, why not contribute back to the project? It’s easier than you think!

Places to start: Spacemacs guidelines for contributors and this great tutorial by Tristan Hume.

Considering you followed the official installation instructions:

git clone --recursive https://github.com/syl20bnr/spacemacs ~/.emacs.d

You should:

• Fork the original repo in Github

• Add your repository as a remote

  If you are using Magit (and you should), press ‘M a’ to add your repo. Magit will prompt for the remote name (I chose ‘my-fork’) and URL.

• Branch from the ‘develop’ branch

  I usually end up with two branches - one that I merge with my own version of master (so that I can use it with the stable version of Spacemacs) and another for PR’ing in the official Spacemacs repository (the ‘develop’ branch).

• Make your changes and commit them

  Spacemacs’ policy is that PRs must contain only ONE commit. No worries - develop with as many commits as you want, and let Magit do the hard work for you by rebasing your branch with the ‘squash’ option. This also works if you need to make changes during code review - just squash your new commits and push them to your remote repository (git will probably complain about the remote tip being behind your branch, but it’s usually safe to ignore this).

• Push the changes to your remote repository and then make a pull request to the original Spacemacs repository

  Remember, all pull requests should be against the ‘develop’ branch, NOT master!

If during this process your branch gets behind the current develop (Spacemacs is a very active project, and as of the time of this writing this happened with me), just press ‘F -r o origin/develop’ in Magit, which translates to

git pull --rebase origin/develop

That’s it, you contributed to Spacemacs and made it one step closer to the best out-of-the-box Emacs experience there is.

And if you are a JS developer, check out skewer-mode’s demo video, it’s awesome!

Vi keybindings in terminal and ansi-term

I did not see this problem described somewhere else, so this is a small post to document it. I was having an issue similar to the one described in this blog post, but ansi-term could print the “weird” characters that were supposed to cause the problem. After checking my dotfiles a bit, I found this piece of code:

# vi keybindings for zsh
bindkey -v
zle-keymap-select () {
    if [ $TERM = "screen" ]; then
        if [ $KEYMAP = vicmd ]; then
            echo -ne '\033P\033]12;#ff6565\007\033\\'
        else
            echo -ne '\033P\033]12;grey\007\033\\'
        fi
    elif [ $TERM != "linux" ]; then
        if [ $KEYMAP = vicmd ]; then
            echo -ne "\033]12;#ff6565\007"
        else
            echo -ne "\033]12;grey\007"
        fi
    fi
}; zle -N zle-keymap-select
zle-line-init () {
    zle -K viins
    if [ $TERM = "screen" ]; then
        echo -ne '\033P\033]12;grey\007\033\\'
    elif [ $TERM != "linux" ]; then
        echo -ne "\033]12;grey\007"
    fi
    zle vi-cmd-mode
}; zle -N zle-line-init

This piece sets up a red cursor for when I am in ‘vim mode’ in zsh. Apparently Emacs did not recognize these escape sequences and generated this weird output:

ansi-term sets up an env variable called $EMACS, so I just wrapped this code around it:

if [ -n "$EMACS" ]; then
    echo "no vi keybindings inside emacs terms"
else
    # the previous giant ball of code is now here
fi

And it works correctly now:

Migrating an Android build system to Gradle

At my current employer, our team recently underwent a migration of our Android app’s build system to Gradle, now the official choice for Android Studio. Following is a summary of the good and bad things in this change.

The good

Automatic dependency management

This feature is what made us consider switching in the first place. Even though we don’t use many external dependencies, including 3rd-party libraries for our project was like this:

• find the project’s source code on Github and download it
• if it builds and runs correctly in Android Studio, set it as a git submodule of our project
• integrate it with Apache Ant, our old build system
• debug and fix any problems Jenkins might have with the new setup

This is far from ideal. In contrast, Gradle got us covered with one line of code:

compile 'your.library.name:major.minor.patch'

Want to package some subdirectories as subprojects? Just add a build.gradle to each of these subdirectories, include them as projects in your main build.gradle and you are set. This encourages modularity and code reuse, always a Good Thing®.

Support for multiple app versions - debug/release, free/paid…

This is the other “killer feature” of Gradle. It is very easy to make different build profiles for an app, each with its own set of source files, compile options, feature toggles and many more. I find it particularly satisfying to know I can stuff my debug builds with instrumentation code and libraries for inspection, and when releasing, no space or processing time will be used.

Other benefits

In the first place, it is very refreshing to use an actual programming language (Groovy) instead of XML. Easier to understand and write, and much better for newcomers not used to Ant (like me - I had to look up documentation every time I wanted to tweak our old build system). This also lowers the barrier of adoption for potential project contributors.

Gradle also introduced the ability to inject constants in the BuildConfig classes of the project. This is particularly useful when injecting API URLs and access configurations (e.g.: extra header codes) in your application. Nothing world-changing, but very welcome.

Did you know that Gradle integration with Ant is excellent, even featuring a one-liner for importing build.xml?

ant.importBuild 'build.xml'

The bad

Android Studio integration issues

Android Studio integration with Gradle is good - if you start a project following its conventions. In our case, we migrated an Eclipse/Ant application, using non-standard paths, which led to unexpected behavior in the IDE. Some team members even reported finding “won’t fix” bugs with non-standard paths.

The GUI for build configuration is very simple, and sometimes it rewrote our build.gradle badly. I recommend always writing it by hand if possible.

Build times

tl, dr: always enable the Gradle daemon. If you only use Android Studio, it’s on by default; if you are using Gradle in the command-line, run this command:

touch ~/.gradle/gradle.properties && echo "org.gradle.daemon=true" >> ~/.gradle/gradle.properties

In my machine, I see an improvement of a few seconds in build times. Which is great, because personally, I find the edit-compile-run cycle in Android slow…but it may only be my appreciation for REPLs talking. Unfortunately, there are none for Android because of technical limitations, but there are plugins that come close and smart workarounds.

Conclusion

There are many benefits and very few drawbacks to using Gradle as a build system (and in my experience, these are mostly related to IDE integration, not issues with Gradle per se). So, if you are in a JVM environment, give it a go! It works for most Java projects, not only Android ones, and there are many plugins to help you achieve what you want, so try it already!