The Wayback Machine - https://web.archive.org/web/20220421113420/https://metaredux.com/

Posts

  • Apr 21, 2022

    RuboCop Turns 10

    2022 has been a pretty bad year for me so far. Still, even in these hard times one can find things to celebrate. On this very day 10 years ago I’ve created RuboCop, a linter and formatter for Ruby.

    Every Journey Has A First Step. Every Saga Has A Beginning.

    – Star Wars Episode I: The Phantom Menace, Trailer

    The first step (commit) for RuboCop was quite modest. Back then I knew next to nothing about parsers, lexers, abstract syntax trees and all that jazz. I recall I planned to implement RuboCop in terms of regular expressions… I remember I had the idea about the project for quite a while, but really struggled to come up with a name that I liked. Picking the name is probably the hardest part of every project and this was certainly true with RuboCop.

    The journey that followed it was epic and beyond anything I could have ever imagined:

    • 201 releases
    • 800+ contributors
    • 220+ million downloads

    That’s an average rate of 20 releases per year over the course of an entire decade! I can think of no moment in time when RuboCop had stagnated and the development wasn’t moving at a brisk pace.

    As I write this RuboCop is at #82 in the list of most download Ruby gems! A project that I’ve started with very modest goals became extremely popular within our Ruby community. Impossible is nothing, right? For me RuboCop is the very embodiment of the power of open-source software and open-source communities.

    This won’t be a long article about the history of the project. I’m way too lazy to write one, plus I already spoke a few times about the (early) history of RuboCop in the past.1 This won’t be a long article at all. Just a small celebration of RuboCop’s 10th anniversary. I don’t know about you, but to me 10 years in software is like 50 years in some other industry. Maybe even more.

    I’m extremely proud of the RuboCop journey so far and all of our small victories along the way. I’m proud of the community that has formed around RuboCop and of the RuboCop Core Team members that have been essential to its long-term success. I’m proud of all of the lessons that I learned along the way. I’m proud of all the challenges that we overcame together. There were many moments when I was ready to throw in the towel and call it a day, but somehow I managed to pull threw. Working on a project of such scale can be extremely emotionally draining.

    I know that RuboCop has become a very polarizing project in the community. In surveys it’s often near the top in both “the most loved” and “the most hated” Ruby tool categories. Some people have told me that RuboCop played some part in their decision to stop doing Ruby. Admittedly we might have gone overboard at times in our desire to improve Ruby codebases.2 We’ve been accused of destroying some of the fun of programming by imposing too many rules. I guess there’s some truth to this.

    But we’ve always listened to our community - a few years ago we promised that with RuboCop 1.0 backward compatibility would become a top priority and we kept our word. We promised that we’d limit the work that people need to do on upgrades and we kept our word. We always listen and your feedback is always welcome!

    Anyways, back to the celebration! To commemorate this glorious occasion we’ve just released RuboCop 1.28, aka “RuboCop: The Anniversary Edition”. It’s packed with more goodies than usual and it has a most auspicious and round version number.3 I just hope we didn’t pack more bugs than usual as well, but I guess you’ll let us know about this soon enough.

    What about the future of RuboCop? Clearly there are always things to improve and we definitely have a few ideas about the journey to RuboCop 2.0:

    • Make RuboCop significantly faster (e.g. 2 times faster than it currently is). Think of it as our version of Ruby 3x3.
    • Organize cops into some “presets” that would address the common complaint that RuboCop checks for way too many things out of the box. Think of this as something like an “essential”, “standard”, “community style guide”, “github”, “rails”, “all-in” and so on configuration bundles that enable different cops. It’s not like you can’t do this today, but probably shipping this out-of-the-box would simplify the setup for many people. There are plenty of interesting ideas to explore in this direction.
    • Internal consistency. Make the configuration uniform and more intuitive across the board. I have to admit that in the early days of RuboCop I didn’t think much about configuration consistency, which was a major mistake on my part.

    I’m confident that regardless of the exact details RuboCop has another bright decade ahead of it. I can only hope that I’ll be able to see it through. When I’ve started the project I was 27 years old, today I’m 37. Time flies really fast. When I was younger I was quite skeptical I’d be able to be an effective programmer after the age of 40, knowing how dynamic our industry is and how things are constantly changing. Still, I love programming just as much as I always did, I love working on open-source projects as much as I always did and I’m cautiously optimistic about powering through another decade of fun OSS adventures.

    And that’s a wrap! Thanks to all of the people who’ve contributed to RuboCop over the years in one way or another! Thanks to all the people who used RuboCop and have been championing it. A huge thanks to the members of RuboCop’s Core Team - you’re all legends as far as I’m concerned and you know it! Thanks to Matz for creating Ruby in the first place! No success happens in a vacuum and there are always a ton of people behind every success story.

    Role models are important.

    – Officer Alex J. Murphy / RoboCop

    Those words have always been the driving force behind the community Ruby style guide and RuboCop itself. I believe that today role models are just as important as they were a decade ago. Here’s to another decade of RuboCop being a great role model for Rubyists out there! Cheers!

    1. https://www.youtube.com/watch?v=nrHjVCuVsGA&t ↩

    2. Read this as “there are many cops checking for things that are not very important”. ↩

    3. At least for software engineers, as 128 is 2^7. ↩

  • Jan 20, 2022

    Weird Ruby: Heredoc Delimiters

    Remember how I thought that the syntax for single-quoted heredocs was weird? I’ve got something even weirder for you today!

    Turns out that when using single-quoted heredoc delimiters you can have pretty much anything in them, including spaces!

    <<'END HTML'
  <html>
    ...
  </html>
END HTML

<<'\n!"£$%^&*()-=_+[];#{}:@~,./<>?|`\'
hello
\n!"£$%^&*()-=_+[];#{}:@~,./<>?|`\

    It never crossed my mind that something like this would be possible until I came across this RuboCop ticket, asking for spaces in delimiters to be allowed. As most editors are unlikely to handle this properly1, and it looks pretty… weird, I think that you should restrain yourselves from using such heredoc delimiters.

    That’s all I have for you today. Keep Ruby Weird!

    Articles in the Series

    1. Clearly rouge (the code highlighter my blog uses) doesn’t handle such delimiters properly as well. ↩

  • Jan 1, 2022

    (reduce summarize year-2021)

    Last year was pretty bad for everyone. So bad, that I didn’t really feel like writing the traditional summary article about it. 2021 wasn’t exactly great either, but it was a relatively good year for me all things considered.

    Below I’ll quickly go over the highlights of the year for my OSS projects. I wrote a separate blog post about everything else here.

    RuboCop

    RuboCop had a pretty good year. We’ve started the year at version 1.7 and we’ve had a total of 30 releases in 2021!

    Nothing really ground-breaking happened - we kept fixing bugs, adding new cops and improving the existing ones. Stability was the key word this year and we’ve been very committed to it. There are no grand features on the horizon and our focus is unlikely to change in the foreseeable future.

    One item that I want us to tackle in 2022 is performance (improvements). Perhaps we need an ambitious goal like “RuboCop 2x2”? Make RuboCop 2.0 2 times faster than RuboCop 1.x!

    A couple of smaller items:

    • RuboCop has a new home on GitHub (https://github.com/rubocop/rubocop) (we finally managed to get the rubocop org and moved away from rubocop-hq)
    • RuboCop now has GitHub discussions enabled (although few people have made use of them)
    • RuboCop now has a Discord server

    nREPL

    I’m really glad that we managed to release nREPL 0.9 a few weeks ago. Now, I guess the focus will become solidifying the public API (mostly that of the built-in ops), improving the end-user documentation and cutting the long-overdue nREPL 1.0 release.

    nREPL is definitely at a good place and I think that by now we’ve achieved all the goals that we set out to achieve when we moved the project out of Clojure Contrib in early 2018.1 Frankly speaking, we’ve already exceeded those original goals. A big thanks to everyone who was a part of this amazing journey!

    At some point we’ll have to document the nREPL protocol better as well, as it’s currently easy to confuse the protocol with the reference Clojure implementation. Perhaps 2022 will be the year of awesome nREPL documentation?

    CIDER’s Orchard

    CIDER had a couple of solid releases this year - 1.1 and 1.2. I’m particularly proud of all the work that went into CIDER 1.2 and I consider it the most important CIDER release in recent years.

    Some other highlights from the Orchard:

    • The new library enrich-classpath made it very easy to work with Java sources and Javadoc. It currently supports only Leiningen, but I hope this will change in 2022.
    • We finally dropped dynapath from Orchard 0.8, which means that Orchard is now dep-free!
    • cider-nrepl got a few small releases throughout the year (mostly bug-fixes)
    • clj-refactor.el and refactor-nrepl got a ton of love (and releases) in 2021 - I think they saw more activity in 2021 than in the past 5 years combined!

    Most of the Clojure Emacs packages are now available for installation via NonGNU ELPA. That’s as close as we can get to out-of-the-box support for Clojure in Emacs, as this repository is enabled by default in the upcoming Emacs 28. This mean you can just do something like M-x package-install cider without any additional setup.

    The only kind of forgotten package in 2021 was sayid, but I hope that this is going to change in 2022.

    Everything Else

    Sadly, I didn’t have much time for all my other projects. I really wanted to work a bit more on Projectile and Prelude, but alas… At least things there are always moving forward, albeit slowly. By the way, did I mention that most of my Emacs projects (not just the ones related to Clojure) are now available on Emacs’s new built-in repository NonGNU ELPA? Projectile, crux, super-save, etc - you’ll find them all there.

    All the style guides had a quiet year as well and that’s fine. It’s not like the best practices change every day. Still, all of them could use more copy editing, clear rationale and more/better examples. No rush, though.

    Discord

    As outlined here I’ve been trying to use Discord as a support channel for my OSS projects. So far the results have been a bit underwhelming, but I’m still optimistic that Discord can be a valuable tool for OSS communities.

    CIDER is probably the only “success story” to date (with about 200 people in the CIDER Discord server), but this still pales to the number of people in the Clojurians Slack.

    Funding

    The donations I’ve been receiving for my OSS projects have been more or less steady for the past 3 years (e.g. CIDER has pretty much the same donations on OpenCollective today as it had in early 2019). OpenCollective and GitHub Sponsors gave some initial boost to donations in 2018-2019, but there was definitely no steady uptrend in the donations in the years since.

    Of course, from time to time there are notable exceptions like:

    • Cognitect donating $500/month for my work on CIDER & friends
    • Clojurists Together’s annual funding of $1500/month for CIDER
    • Occasional bigger one-time donation

    Still, the dream of working full-time on OSS projects down the road seems to be as elusive as always. From time to time I come across idealistic articles like Funding isn’t the Problem with Open-Source, and I certainly can’t agree with them. For one, I’ve yet to encounter those evil corporations that are willing to pay me money to subvert my projects to their bidding (and evil ways, of course). Topic for some dedicated article I guess.

    I admire a lot people like Michiel Borkent, who recently went all-in on his OSS work. They give me hope that everything is possible if you put your mind to it. Perhaps I’ll be ready to take the red pill a few years down the line myself. Especially if I make the right crypto “investments” in the mean time.2

    I’d like to give a big thanks to all my patrons! Your support means a lot to me and keeps me going after all those years! You rock!

    Changes to Meta Redux

    You might have noticed that this year I’ve been writing fewer articles here and they are mostly about my OSS projects. That’s related my decision to revive my original site (think) and do more of my writing there. And give some clear focus to Meta Redux by doing son.

    Epilogue

    2021 marked the 10-year anniversaries of my oldest projects Projectile and Emacs Prelude. A decade working on OSS projects! Wow, time is certainly not on my side, as I have ideas for at least 50 years of work…

    I don’t have any grand plans for 2022 right now. Clearly my focus will stay with CIDER and RuboCop, but I hope that I’ll manage to find some time for my smaller projects as well. One thing is certain, though - I love hacking on OSS projects just as much as I always did and I’m always looking for ways to take things to the next level. Happy New Year! Keep hacking!

    1. https://github.com/nrepl/nrepl/issues/1 ↩

    2. Just kidding. I hate crypto, but many of my friends seem to believe that’s the quickest way to wealth and prosperity. ↩

  • Dec 22, 2021

    CIDER 1.2 (Nice)

    Unfortunately, no one can be told what CIDER is, you have to see it for yourself.

    – Clorpheus, The REPLix

    Today is the day. 22.12.21. 12/22/21. No matter how you cut it, it’s one auspicious date and if it’s good enough for a new Matrix movie, it’s certainly good enough for a new CIDER release as well! So, it is with great pleasure that I announce the immediate release of CIDER 1.2 (“Nice”). We have been brewing it for 7 months, but I can assure you that the long wait was worth it.

    First, an explanation of the name. I’ve spent the final weeks of my sabbatical from work in Nice and Antibes and I totally loved my time there. Admittedly I didn’t do any programming at all in France (I didn’t even take my computer with me), but it’s there that the real work towards CIDER 1.2 began. I spent a lot of time today wondering whether to name this release “Nice”, “Antibes” or “Côte d’Azur”, before settling on “Nice”. Naming is hard!

    CIDER 1.2 is not just Nice! It’s f*cking Great! It’s easily the most ambitious release in the past 3 years!1 Here’s why…

    Highlights

    Dynamic nREPL Connection Upgrade

    Ever since in 2014 (CIDER 0.7) I made the decision to make some of CIDER’s functionality depending on cider-nrepl, some people have been frustrated that CIDER effectively needed a special nREPL server to unleash its full potential. In practice that meant additional setup that sometimes wasn’t even possible (e.g. you usually can’t add development dependencies to production apps).

    CIDER 0.11 (released in 2016) addressed part of the problem, by coming up with the notion of jack-in dependencies that were automatically injected. Down the road the problem with remote REPLs was alleviated somewhat by extending the nREPL protocol with ops like lookup and completions. Still, there was no easy way to just connect to a regular nREPL server and teach it all the tricks that CIDER needs it to know.

    Today this changes. Just run M-x cider-upgrade-nrepl-connection after cider-connect and magic will happen! This (experimental) feature is the culmination of a lot of work that was happening in nREPL, over the past couple of years, to allow clients to dynamically load code in nREPL (a feature known as sideloading).

    Huge thanks to Arne Brasseur for working on this and for all of his contributions to the Clojure community over the years! Arne wrote a couple of nice articles on the subject that I can heartily recommend:

    The current connection upgrade approach is not without its flaws, but I’m confident we’ll be able to improve it a lot down the line. As Arne mentions we might be able to find simpler alternative approaches as well and that’s totally fine.

    By the way, did I mention that now nREPL’s sideloader is fully supported in CIDER, so you can load random Clojure libraries (not just nREPL middleware) on demand? Here’s how it works:

    • Place some resources on cider-sideloader-path (e.g. a cider/rocks.clj file)
    • M-x cider-sideloader-start
    • require cider/rocks somewhere and it will be loaded on demand

    I just realized I should write documentation on the subject. Oh well, at least we have the official nREPL docs and the blog post I wrote about sideloading a while ago.

    Auto-fetch Java Sources and Javadocs (Lein-only)

    I don’t know how many of you have used Maven, but one thing I loved about it that it could just fetch all the Java sources and Javadocs for the packages I used in my projects and then I could easily navigate to definitions, read Javadocs locally, etc.

    CIDER has had support for parsing and navigating to Java sources and Javadocs for ages, but it required users to manually ensure the necessary resources are on the classpath. In CIDER 1.2 we finally catch up to the Maven experience, thanks to enrich-classpath - a library (and Leiningen plugin) that, as its main feature, automatically downloads all available .jars with Java sources and javadocs for a given project, so that various tooling (e.g. CIDER) can access it.

    This was supposed to be enabled by default, but we hit a mysterious last-minute Lein issue, so you’ll have to enable it yourself if you want to try it out:

    (setq cider-enrich-classpath t)

    When this is enabled (and working), the experience is magical - you can navigate to any Java source with M-. without any additional setup!

    Note: Our debugging of the mysterious issue leads us to believe it’s a simple case of not accounting for the JDK sources and docs being installed. On Debian-like Linux distros that is as simple as:

    $ sudo apt install openjdk-11-source openjdk-11-doc

    On Fedora/Red Hat you’ll need to do something like:

    $ sudo dnf install java-11-openjdk-src java-11-openjdk-javadoc

    I encourage all Leiningen users to enable the enrich-classpath integration and share their feedback about it with us!

    Another round of huge thanks for vemv, who was the driving force behind enrich-classpath!

    cider-jack-in support for babashka

    The title here says it all. Basically, CIDER now knows about bb.edn (same way it knows about project.clj and deps.edn) and will launch a babashka REPL if it detects one.

    This simple feature took a while to brew, but it was also a proof that hammock-time really works, as the final version is much simpler than anything else that was original considered/proposed.

    Up next - nbb support!

    Unix Socket Support

    nREPL 0.9 supports listening to Unix domain sockets, and now CIDER supports connecting to those. When using cider-connect just pick local-unix-domain-socket as the hostname (it’s kind of magical) and then type the path to the local socket file. Don’t forget that you’ll need to start nREPL a bit differently:

    $ clj -M:nREPL -m nrepl.cmdline --socket path/to/nrepl.sock

    The support for this in CIDER is experimental and subject to changes. Down the road it will be able to jack-in using an Unix socket and we might have some auto-detection for their presence in cider-connect. As usual - sky is the limit and your help is most welcome!

    xref

    xref is an Emacs framework for looking up identifiers, that was introduced in Emacs 25. CIDER provides pretty much all of its features (e.f. find definition, apropos, find references, etc), but because it didn’t have an xref backend it was overriding a lot of its keybindings (e.g. M-.) and that frustrated users of xref.

    Beginning with version 1.2.0, CIDER supports Emacs’s built-in xref functionality, which means M-. will invoke xref-find-definitions instead of CIDER’s own command cider-find-var. You can disable the use of CIDER’s xref backend like this:

    (setq cider-use-xref nil)

    Note: You’ll have to disable and enable cider-mode for this setting to have effect.

    If you use other packages that also integrate with xref (e.g. lsp-mode), you may wish to customize the precedence of CIDER’s xref backend. The precedence is controlled by the order in which backend functions appear in the xref-backend-functions hook. By default, the CIDER xref function will be added with a depth of -90, so it will (should?) come first. If you would prefer for it to have a lower precedence, you can change cider-xref-fn-depth:

    (setq cider-xref-fn-depth 90)

    See Setting Hooks for more information about depth.

    Everything Else

    As usual there are plenty of small improvements and plenty of bug-fixes. I love the richer completion annotations in company-mode and the usability improvements for ClojureDocs.

    Some bug-fixes, like handling of empty stackframes, have been long overdue. Better late than never!

    cider-nrepl and orchard got a lot of love as well, and I’m happy to report that in Orchard 0.8 we’ve removed our last runtime dependency (dynapath), and now Orchard is completely self-contained. What’s even better - it does no classpath manipulations at all, for the sake of putting tools.jar and your JDK’s sources on the classpath. enrich-classpath is the way to go.

    Next

    Our work is never done. Our roadmap is never empty. Our open issues keep growing. As usual this releases ended up taking more time than I expected it to take and the final scope is a bit smaller than what I had in mind. I’m not concerned about this, as there’s always the next release.

    I plan to start distributing CIDER over NonGNU ELPA soon, so it’s available for installation out-of-the-box for everyone using Emacs 28+. clojure-mode and inf-clojure are already available there.

    I guess improving the sideloading experience/connection upgrade will be one of the focal points of the next CIDER release as well. ClojureScript support continues to be an area that needs a lot of love, as does connection management with sesman.

    We’ll need to figure out how to bring the awesomeness of enrich-classpath to tools.deps users. Your help is most welcome!

    Epilogue

    And that’s a wrap! I can’t remember when was the last time I wrote a release announcement that long.

    As usual, here I’ll thank all the people who contributed to the release in one way or another. You rock and you show why the Clojure community is so awesome! Another round of special thanks goes to Clojurists Together and Cognitect, the main patrons of my open-source Clojure work!

    2021 was another interesting and pretty tough year. Conference-driven development is dead, but at least Matrix-driven development is making a comeback. Clojurists together STRONG!

    Happy holidays, everyone! CIDER 1.2 is my small Christmas gift to the Clojure community! I miss you all and I hope that next year will bring us back to together! I’m running out of sanity, but I’m never running out of Emacs. M-x forever!

    1. Don’t panic! Upgrades should be painless! ↩

  • Dec 12, 2021

    nREPL 0.9

    nREPL 0.9 is out! We’ve been working on it for over a year, but we finally got where we wanted to be.

    The highlight of the release is undoubtedly the support for Unix file sockets, which are a secure and fast alternative to TCP sockets for local development. This feature was on the books for a very long time, but it became viable only recently when Java 16 added support for Unix sockets.1 If you’re stuck with an older version of Java we’ve got you covered as well - just add junixsocket as a dependency and nREPL will pick it up automatically. With tools.deps you can do something like:

    {
;; ...
:aliases {:nREPL
          {:extra-deps
           ;; UNIX domain socket support was added in 0.9.0
           {nrepl/nrepl {:mvn/version "0.9.0"}
            com.kohlschutter.junixsocket/junixsocket-core {:mvn/version "2.3.2"}}}}
}

    Afterwards you can start nREPL like this:

    $ clj -M:nREPL -m nrepl.cmdline --socket nrepl-socket

    CIDER has already added experimental support for Unix sockets and I hope that other nREPL clients will follow suit soon.2 The changes necessary to clients are tiny, as only the connection mechanism changes, everything else stays exactly the same as with TCP sockets. The magic happens in nrepl.socket, which serves as a compatibility layer for java.io and java.nio.

    If you’re not familiar with Unix sockets you might be wondering if this is something that is relevant for you. I’ll try to explain the impact of Unix sockets here briefly:

    • UNIX domain sockets know that they’re running locally, so they can avoid some checks and operations (like routing), which makes them faster and lighter than TCP/IP sockets.
    • You can’t accidentally start an nREPL server that’s listening on a public address and allows external parties to execute any code on your computer.
    • Security/access is enforced via file permissions.
    • For clients it’s easier to start a server and connect to it, because they don’t have to deal with a random port that they have to figure out once the server starts. E.g. cider-jack-in can just select some socket name based on the current project and connect to it immediately.
    • They are not supported on Windows. No surprise here given their name, right?3

    In practice all of this means that Unix sockets are generally a better option for local development than TCP sockets. You’re free to keep using TCP sockets, though, and they will always be the default in nREPL.

    The 0.9 release also brings a bunch of small improvements and bug-fixes. One more thing - now it’s possible to use nREPL with GraalVM native images. Read the release notes for all the details.

    As usual, here I’ll thank all the people who contributed to the release in one way or another. You rock and you show why the Clojure community is so awesome! Another round of special thanks goes to Clojurists Together and Cognitect, the main patrons of my open-source Clojure work!

    What’s next? Time will tell. It seems that nREPL is feature-complete at this point and that we’ve made good on all the promises we made several years ago when nREPL’s development moved away from Clojure Contrib. Other than polishing the existing functionality, the nREPL protocol specification and the documentation there’s nothing major I want to tackle at this point. Perhaps it’s time for a 1.0 release?

    In parentheses we trust!

    1. Remember that nREPL can’t have any runtime dependencies, so we’re limitted to what’s available in the JDK and Clojure. ↩

    2. See https://github.com/clojure-emacs/cider/pull/3088 for more details. ↩

    3. It was pointed out to me that these days Windows actually supports Unix sockets! See https://devblogs.microsoft.com/commandline/af_unix-comes-to-windows/ for more details. ↩

Subscribe via RSS | View Older Posts