On To My Next Adventure

Taken by a co-worker when App.net was still picplz.

I’m going to assume you have read the news about App.net. It’s clearly the context for this post, but it’s not the point. I am leaving App.net to be the VP of Engineering at new company that should have a name shortly.

Durring my time at App.net I constantly operated at the edge of my abilities, and past them sometimes. It was alternatingly painful and exhilirating. I will be processing my stay here for quite awhile, but the main thing I learned from App.net, that one thing I will carry forever is the relentless pursuit of the problem.

The problem shifts over time. It can be a bug in the code base, a support request, or a product decision but to fix it and to fix it right requires a relentles pursuit of the fundamental problem in question. Once you truly know what the problem is you can fix it, and fix so it doesn’t come back.

It’s not something that everyone learns as an engineer or otherwise, but engineers benfit from it greatly.

So, time to start my next adventure.

How Vox Media Creates News Products

Yesterday we got a look into how new news organization are working in a technical sense1. To my mind its one of the few detailed posts about how places like Vox Media are working to actually innovate. It was written by Pablo Mercado, Vox Media’s VP of Technology. He detailed a little bit about how they pull together all the talent needed for a project and how they coordinate that talent. From writer, editor, designer, to programmer.

New news orgs fascinate me, because they are the perfect collision of technology and domain expertise. In a sense, if we can improve journalism with technology then we can use technology to improve other fields. I’m also not talking about software eating journalism, but working with journalism to make its better, and maybe even pay for it’s self.

I don’t know about you, but I want to read more stories like this. There doesn’t seem to be a great place to find stuff like this. So, if you know any leave a comment.

Try Out The App.net Comments Widget

A week ago (maybe two), we had an App.net hackathon where I showed off an App.net based coments widget. Since then a couple of us have been working an official version of the widget. Its now in a state where I would like to start beta testing it. If you would like to test it out PM me on App.net (@voidfiles). I have also embedded the current version on this page.

All the things: Paper vs Unread

When Paper and Unread launch in the same week its hard for me to not compare the two. The obvious comparison is that they are both news readers1. Its like they both answer the same question with wildly different answers. Paper is all the things to all the people, because Facebook. Where as Unread uses the same things to make something equally spectacular, but for far less people. As much as I wish I could teach everyone in the land to use a reader (because its better), they aren’t going to listen to me.

Secondly, it’s exciting just to watch people invent the ways we are going to interact with our world. Both apps have a great native touch experience. The people who made these two apps understand their medium. Think about what this says about software, innovation, iOS … blah blah, Jared Sinclair is one dude and his apps are as good as Facebooks’. He understands the medium just as well as they do and because of it his apps feels just as innovative.

So here’s what I am going to do. Buy the app for 2.99, pay 30 dollars a year for Feedbin, and another 24 dollars a year to Newsblur because I want these things to exist.

Hopefully as long as enough of us continue to think they are worth our money they will continue to exist. And these lone individuals will continue to listen to the crazy people, the people who read (a lot), because they love the things we love.

  1. reader is a funny word to use to talk about Paper, but thats what it is.

A Newspaper of Images

While looking at the launch of Paper I was struck by the lack of words. The apps name, presumably an homage, suggests a certain sense of wordiness but instead everything is focused on images.

Paper is certainly picking up on trends built by popular apps like Flipboard, and it’s entirely possible that what people want is an even more visual experience, but you loose something when you depend so largely on images. The average blog post is going to look horrible next to the glossy layouts. Proabably adding one more nail the lone blogger coffin.

Look I don’t fear for the literacy of our future selves or anything like that, but if you are looking for a better news reading app this probably won’t be it. Even if it’s is a huge success, there will still need to be a place for words.

Comet — My ADN hack project

Today is an App.net Hack day and my project is an embeddable comments widget that is built on top App.net Posts. It’s actually embedded on this page. So, feel free to test it out and give me feedback @voidfiles. You will need an App.net account to see the comments (for now), and you will need one to leave a comment for sure.

If you are intrested in using this widget your self, just embed this code somewhere on a webpage. There is one bug right now, on mobile safari the browser is getting scrolled to the top of the comments, but other then that it should work.

This is alpha So the things like the url will change.

I’m the newest contributor to The Changelog

I am a big fan of The Changelog. There is a folder in my reader dedicated to raw sources just so I can find new open source projects, but it was always nice to read The Changelog because it was curated. When I finished the first version of my brand new open source project, Lark, the first site I thought to contact was The Changelog.

Over the Christmas break I aproached them to see if they would be interested in covering not just Lark but a few other open source projects. That initial contact evolved into a disscussion about covering the open source community in general and eventually we talked about the prospect of having me join. I jumped at the opportunity. After talking with Adam and Jerod about it, and the future of the project I am thoroughly excited to be apart of what they are doing. If you don’t already subscribe to The Changelog you should.

My first contribution went up this morning its about Sandman: Give any SQL database a REST interface with Sandman

FeedShare: an OPML sharing service

Ask, and ye shall recieve. Brent Simmons posted his OPML file on saturday1, and also wrote “I wish there were an OPML sharing service.”. I am sure more than a handfull of people thought, I can do that, but FeedShare beat us all 2.

I added my feeds and I hope you do to.

  1. http://inessential.com/2014/01/04/my_feeds

  2. http://inessential.com/2014/01/06/feedshare_net

Introducing Lark a RESTy interface for Redis

Lark is a python library that provides a generic method for transforming a HTTP request into a redis command. If you know what webdis is then you’ll roughly know what this is. It does a couple of things right now:

  • It users REST as a guideline without getting to pedantic.
  • It has built in support for per user key prefixs.
  • It automatically JSON encodes redis values (where appropriate).
  • It has lots of tests (and TravisCI all setup).

I have been slowly amping up this project so that it could be my holiday project1, because I have a host of things I want to add quickly. Webdis has authentication right out of the box. Lark is setup for a scope based authentication scheme, but there is no examples of really how to do that. This is the next thing I am going to work on creating an authentication/authorization layer that people can use.

So, if you are interested checkout the github page and fork it. Tell me how crazy I am, tell me where this thing just doesn’t work at all like I say it does.

  1. for more on why holidays are a great time to work on projects checkout Macdrifters post Seasonal Tasks

The Story of ADNpy: An App.net API Client

This is the story of ADNpy. A python App.net API client. From the earliest days of the App.net API I wanted to build a client. In the beginning though the approaches were infinite. For as many times as I started an API client, I stopped because I couldn’t figure out how to start.

The fact is though, we are an API company and that meant we had to consume it. We didn’t have the luxury of thinking about our options we just had to start building. Time went on though and in a series of small steps we created a python API client. So, a couple months ago I started taking our pieces and built a client library.

The Parts

In July 2012, we started to work on the API. We decide the best way to showcase the it was to build a client. Thus Alpha, from day one, was going to be an API consumer, but what did that mean for us though?

For one thing we run on Django all be it with many pieces replaced. Our common data access pattern was to make a query to the database using Django’s ORM and to then operate on the models. That method was going to turn into making REST API calls and getting back JSON objects.

This made the first part pretty clear. For the sake of letting old code work roughly the same as the new we wanted to use dot notation to access the properties of our API objects instead of using dict notation. This has the added benefit of being more aesthetically pleasing. The way we ultimately solved this had the added benefit of being able to fallback to to dictionary lookups instead of using hasattr, or getattr (even though they still work as expected).

You can see this piece in adnpy.models.

Our first internal API consumer was Alpha, but it wasn’t comprehensive, Alpha doesn’t do everything the API can do. As we developed the API we all ended up building ways to make calls to the API for testing purposes. Besides needing to use the a dev, sandbox, and production version of the API for work, I build a lot of little test projects.

In my case, I built a script that modified a requests session that would make calls to the API that looked like this, r.get('/posts'). This is prefect for someone who has a good understanding of the API and all its paths, but not so much for a new comer. I needed to add a way to go from a class method, to a path plus additional configuration information. You can see the descedant of that code in the API handler.

The Build

After many months of frustration that I couldn’t have the best features of our internal API client in an easily installable python package. I decided to pull it all together in one python package and release it.

My goal was to build a python package in the most modern way possible with tests and nice-to-use API.

Seeing as that I use requests I heavily copied Kenneth Reitz style of python package. His guide to releasing python projects has also been incredibly helpful.

Considering python API’s, I have always enjoyed using tweepy when I need to interact with the Twitter API, so I was very influenced by how the tweepy API worked. I liked that you could get a User object, and you could then call methods on that User object. Another great idea in tweepy was the cursor object. Even though I consume the API all the time I would rather not think about pagination when I don’t have to. So, you can see that ADNpy also has a cursor.

I also took note of how tweepy built up its API internally. This allows one to encapsulate all the different configuration options required to make an API request. It also made possible the auto generation of doc strings so that there is at least some API docs, without needing to hand write each one. You can see the docs here.

Finally came testing. I wanted something that I could run over and over again with out accidentally tripping the API rate limits. The best would be something that could replay API calls. Happily that exists. This way as I made new API requests they would be recorded to a JSON file and replayed from the JSON file in the future. Using this tool I could run the whole test suite without needing to make actual API calls.

With this all in place I was able todo some TDD for the API. I’m not really a zealot when it comes to any paradigm, but in this case TDD was something that really helped.

The Release

Once, I had the library and the tests it was time to release, but I still needed a few things. I wanted to have some docs with all the main API methods documented and a “quickstart” guide. I also needed to host the docs. Again borrowing ideas from requests I used sphinx for the docs and I hosted them on Read The Docs.

Finally, I wanted there to be some kind of continuous integration. You may have seen some github projects that have these images saying the tests pass. That comes from an incredible open source resource called Travis CI. It will do continuous integration for any public Github project, for free, and it’s integrated into Github using webhooks.

There is a nice flow to all of these tools. I write code and test locally. I update the docs as needed. Then I commit to Github. Github will then ping travis and Read The Docs. Which in turn run the tests and rebuild the docs, so I don’t have to do a thing.

From time to time, I just need to push a new version to pypi.

The Future

The test package I was using to do replay no longer works. I’ll need to figure out how to fix that, until then running the tests often is problematic. I am hoping someone will fix it, but it may have to be something I do.

I am still not satisfied with how the docs have turned out, but its better then nothing. I at least didn’t want people to have to look up the code all the time to figure out how to do things. I think I have achieved that. But, the docs could be a lot more helpful.

In the beginning, I am trying not to have to much of a schedule. I will continue to push point releases until I feel that we have a very strong v1 and then there might be a more stringent release schedule.

At the end of the day, this is just one part of a larger goal which is to make using the App.net API incredibly easy. Libraries like this one are only part of that effort, but I have found that building these kinds of projects expose the holes in our existing efforts to support developers. So, releasing new open source projects that help developers is something I will continue to do. It’s only going to make App.net better.