1. how to start with a reasonable CSS stylesheet that doesn’t look ugly but it’s also not a pain to use
2. a series of highly reusable HTML fragments that do the things we do all the time (like, add jquery to this page, or add a header and a footer, etc)
3. document a series of javascript libraries that enhance the default Acre environment and simplify your life

So, look here:

• http://default.freebaseapps.com/ – contains some templates that create simple HTML fragments for you to use and a default CSS stylesheet that makes your page look reasonable from the get go (the app uses its own default_stylesheet so it will look like that one)
• http://libraries.freebaseapps.com/ – lists useful JS libraries that you can use alongside with example code you can cut/paste to get you going
• http://wiki.freebase.com/wiki/Acre_Recipes – contains useful recipes and code fragments that you can cut/paste in your app to do useful things in acre that are not obvious. Now wikified! so you can add/request your own.

Enjoy!

• dump_base is working again
• in load_base, the target domain is created automatically if it is a sub-namespace of the logged-in user’s domain (thanks praveen)
• some unicode issues in mqlwrite are fixed
• slightly saner logging under the “freebase” logger rather than the root logger
• new ‘application_name’ settable on the MetawebSession which will get inserted into the user agent
• switch to form POST by default to account for long mqlread queries

You can install the library using easy_install freebase from the command line.

Some other interesting tools that are part of the acre-development app include:

• Most-cloned files
• Published apps without a description (are any of yours on there?)
• An RSS feed for the newest apps by creation date.

As usual, the source code for the app is there for you to read or clone.

Quick notes:

Yoz started the show with a 15 minute version of a longer “Folk Programming” talk he does, followed by me talking about how the concepts apply to open source software, and then Jason talking about how we (i.e. Freebase.com) applied the concepts to our new app development platform to help encourage adoption and a strong developer ecosystem.

Yoz starts out talking about he first learnt to program (on a ZX Spectrum), and how beginning programmers often cargo cult as a way of learning. Cargo cult programming is often stigmatised, but it can be seen as a kind of folk process, like folk music or folk stories, and it’s a good way to get started.

He then gave a bunch of examples of folk programming in action, from an MIT study of children passing around little games on Dreamcast modules, to Yahoo Pipes and Second Life. He identifies five things necessary for a real folk programming environment:

• Create your own code.
• View, clone, and modify someone else’s code.
• Free, always-on, ubiquitous hosting.
• Huge array of diverse data sources.
• Code creates new data for use by others.

I then took over and slightly reframed his list for the open source world:

• Free and Open Source licenses
• Easy cloning forking
• Hosted development environments
• Huge array of data

I looked at some examples including github, Amazon EC2, and Google App Engine, and looked at them to see whether they provided folk programming environments for open source development. What I found is that each provides some elements, but none provides all of them.

When Jason and his team were starting work on our hosted app development platform, Acre, we got Yoz in to give a lunchtime talk about Folk Programming, and used a lot of his ideas in developing our platform. Jason walks us through how Acre has BSD licensing, one-click cloning, and interacts with Freebase’s huge store of open data.

Some of the demos he shows include “data games” for Freebase contributors to more quickly add data to Freebase itself, which started off with a single app called Typewriter and quickly evolved into a cluster of apps and eventually a library and toolkit to allow people to quickly build more. (Some were built at our recent Hack Day, for example, including one to easily find CC-licensed images to attach to Freebase topics.)

Another set of apps he showed were clustered around the concept of lists in Freebase, starting with an app to create lists of anything and then showing a recommendation website and an app for TV fans all based on the same underlying data.

Jason closed by talking about some of the challenges we’d faced and future directions for folk computing in the open source world. Challenges included version control and collaboration (because there is a time when you’re no longer experimenting and need to work together on things rather than fork), and the state of in-browser development tools like syntax-highlighting code editors and Firebug-style console debugging for server-side apps.

Questions from the audience included “Have you seen people writing large apps in folk programming environments, in the thousands or tens of thousands of lines range?” to which the answer is “yes” — LambdaMOO and SecondLife being enormous, and some of the Freebase apps being in that linecount range already despite only having properly launched about two weeks ago.

“How do you handle security?” was the second question, which Jason answered at length. Some of the parts of the Acre security model included sandboxing on the server; having each app on its own subdomain to prevent cookie hacks; and everything being sanitised by default to protect against cross-site scripting attacks.

If you haven’t looked at Acre or freebaseapps.com in a while, a lot has changed, and it’s worth a visit.

What’s new:

• The Acre platform itself is officially at version 1.0, which means API stability.
• Our new App Directory makes it easy to find Acre and non-Acre Freebase apps. You can search by app name or author, and our featured apps include some of the best ones from the Freebase developer community.
• Our developer documentation has been completely revamped.
• Our developer tools, including the query editor, app editor, and schema explorer, have been improving by leaps and bounds over the last few months.

Take a look at our screencast, too, to see what’s new:

The freebase-python library 1.0 is now available! It introduces a new syntax for accessing the freebase api, it updates the available commands to reflect the entire web api, and it introduces some cool schema manipulation utilities. It’s backwards compatible with previous versions of the library.

The new syntax makes it easy to start running with freebase:

> > > import freebase
> > > print freebase.mqlread({"id" : "/en/the_beatles", "name" : None})
{u'id': u'/en/the_beatles', u'name': u'The Beatles'}

You can also access freebase.sandbox as easily:

> > > import freebase.sandbox
> > > print freebase.sandbox.mqlread(...)

freebase-python can now do all of the current api calls that the website handles. For a full list of the api calls and examples, you can refer to http://code.google.com/p/freebase-python/wiki/GettingStarted.

The schema manipulation utilities introduced make it easy copy and move types, properties, and objects. This means that you can save bases to your computer so that their survive a sandbox refresh, keep different versions of your bases and types on your own machine, or move types between bases. There are a few recipes on the freebase-python wiki that you can find here: http://code.google.com/p/freebase-python/w/list?can=2&q=label%3ARecipe%2CSchema-Manipulation

freebase-python is also Google App Engine friendly. You can check out how to write your own app at http://code.google.com/p/freebase-python/wiki/GoogleAppEngine. You can also check out a sample app at http://freebase-images.appspot.com.

There are also more test and more docs!

Installing is easy, if you have easy_install, just say

$ easy_install freebase

If you don’t have easy_install, you can get it by running http://peak.telecommunity.com/dist/ez_setup.py

If you’re like me, MQL often feels like a combination of power and dark magic: some people can whip up a MQL query in no time, like they knew all about every type, every property and how they link all the various pieces of the graph together.

Asking around how people did it, yielded several strategies… but many basically revolved around using the ‘explore pages’ (those magically revealed by hitting the F8 key on freebase.com).

Then David wrote the new query editor, with the amazing context completion and I thought I was set… but while my MQL mojo improved a lot with it, it always felt like something was missing: if I knew the type I wanted, I could get started… but what if I didn’t know where to start? what if I didn’t know how to construct the query and I needed some guidance in navigating how the types were all connected together? (especially with CVTs and whatnot)

I felt like I was in a fog and with no map: finding my way out felt tentative at best, frustrating at worst.

The Jason showed me an acre app he had written a long time ago and that he uses all the time that offered a browsable interface to all the freebase domains, types and properties, their IDs… and how they all connect together.

I had found my map.

Unfortunately, while extremely useful already, the schema explorer app needed some love… so I decided to give it some and the result has been so successful internally that I’m happy to announce it to the public so that everybody can benefit from it. Find it at: http://schemas.freebaseapps.com/

I hope you’ll find it as useful as I do.

NOTES:

1) suggestions/criticism/feature-requests appreciated and welcome

2) this is an acre app and if you’re curious (or interested in forking it) you can find it at: http://acre.freebase.com/#app=/user/jdouglas/schemas

3) this couldn’t have been possible without Jason’s work upon which I built this so kudos go to him while all the mistakes remain mine

4) I’ve been working on integrating the schema explorer with the upcoming freebase developer’s documentation hub, so that newcomers won’t have to hunt down this URL in the mail archives to find a little guidance in the type maze.

Enjoy!

This release introduces a considerable amount of improvements in both the app editor and in the Acre engine itself.

APIs cleanup and deprecation

Acre APIs have grown organically, starting by emulating a simple CGI-like model. Over time, we realized that a request/response model was better suited for Acre apps.

This Acre release marks the deprecation of the old CGI-like model (for example, acre.environ.* and acre.context.*) and the official introduction of the new request/response objects (acre.request.* and acre.response.*) which are now the recommended way to interact with the HTTP request/response life cycle.

It is important to note that even if deprecated, all APIs are still available and will remain available for a few more releases: we understand that back-incompatible API transitions are highly disruptive and we want to minimize the impact we’re having on our community of developers. At the same time, we want to avoid disruption later and this is why we’re doing this transition now, before a 1.0 release.

It is worth noting that while Acre will execute apps that use deprecated APIs unchanged, it will also log ‘warning’ deprecation messages in the console (which can be obtained by adding setting the “acre.console=1″ parameter in every URL served by Acre).

We highly recommend following those deprecation warnings to update your apps to the new APIs, so that your app will continue to function in the future once the deprecated APIs will be turned off.

The new documentation hub (mentioned below) shows which API methods and variables have been deprecated and what are the new ones that should be used.

App Directory and Documentation Hub

Acre now has two important new visible parts: the app directory and the documentation hub.

• The app directory is the place for browsing and searching for apps. When you release an app to a short url it will be automatically added to the directory. You can edit its description, or even de-list it, in the General tab of the app menu (click on your app’s name) in the App Editor. Find the app directory at http://freebaseapps.com/apps
• The documentation hub is the place where you can find information about all the Acre APIs, the Freebase APIs and web services, the MQL language, etc. It’s still a work in-progress, but we’re also excited to introduce a new documentation viewer that brings together all of our reference documentation — for Acre, MQL and the Freebase Web Services. It’s both available in App Editor from the help menu, as well as stand-alone at http://freebaseapps.com/docs

App Editor

Several important improvements were made in Acre’s app editor:

• new query editor: guides you thru the creation of a MQL query with context-based auto-completion. It will also generate a simple acre template to generate an HTML page from your query and get you going faster. The keyboard shortcut for invoking “query assist” is the Tab key.
• code completion: app editor is now is capable of providing code-completion functionality and guide you during app development thru the discovery of Acre’s API and its parameters. The keyboard shortcut for invoking “code assist” is Alt-Space on the Mac and Ctrl-Space on other platforms. Code assist also shows up automatically after you type a period after “acre”.
• improved app selection menu: makes it easier to move from one app to another with the list of most recently used apps.
• console goodies: the log console now has a link to the source code of the page being executed and a breakdown of the ‘costs’ of executing the request (useful for profiling).
• no more extra blank lines inserted while cutting and pasting code inside app editor.
• files are now automatically saved when previewing.

NOTE: if you see some weird behavior of the app editor, try ’shift reload’-ing it as some browsers are pretty aggressive with caching parts of the application.

Engine

• allow ${} substitutions in <script> and <style> tags
• error pages have links back to the originating source line
• shift-reload now always returns fresh values

For more information on all the bugs fixed in this release, you can look at the issue tracker change log.

Happy Acre coding!

Some of the new features include:

• Syntax tolerance — if you leave quotes off, or forget a comma, it’ll deal. You can also ask the editor to clean up your syntax for you.
• Query assistance — hit “tab” to open out a list of possible topic IDs, properties, types, etc. Now you don’t need to have a second browser tab open to explore schema!
• Interactive results pane — lets you move around the result set more easily
• Turn queries inside out — change “authors and their books” to “books and their authors”, automatically.
• Automatically generate Acre templates to display your query results
• Variables inside queries, so you can refer to something repeatedly throughout your query and then only have to change it in one place
• Set query envelope headers via a simple form
• See how expensive your query is, and why
• Better organised examples

You can read more about these features on the about page or take a look at this screencast for a demo:

Freebase Query Editor 2.0 from David Huynh on Vimeo.

We just pushed a new release of Acre, our hosted environment for developing Freebase apps and services.

Some highlights of the release:

Short URLs
You can now register a short http://*.freebaseapps.com URL for your app in addition to the http://*.username.user.dev.freebaseapps.com/ URL your app was given when you created it. Registering this “release URL” means your app will also be listed in the up-coming App Directory (soon!). To register a release URL for your app, go to the new Versions tab of the app menu.

App Versions
While there’s much to love about the live editing nature of Acre… it’s not without it’s downsides. How do you share your app with people without having to worry about breaking it as soon as you resume development?

To address this problem, we’ve introduced a new versions feature. Versions are addressable snapshots in time of an Acre app. You create a new version of the current state of your app just by giving it a label. That label can then be used in URLs to always refer to that app as of that moment in time, e.g.:

http://label.appname.username.user.dev.freebaseapps.com/

Note that it’s only the top-level app that is frozen in time. If your app refers to files in other apps, they do not inherit the same timestamp. (If you’re adventurous, you can try out the new versioned acre.require — there’s now a second argument for the label — but it hasn’t been thoroughly tested yet.)

In addition, any version can be mapped to the app’s release URL if you’ve registered one. So you can create a version, set it as the released version, and then continue developing. Once you’re at a new safe point, you can the create another new version, update it as the released version, and your users should be seamlessly upgraded.

New Debug Console
We’ve made a shift in this release to sending logs back down in HTTP headers (when the x-acre-enable-log request header is present in the request). This change should be transparent other than the fact that the web-based console is now triggered by the presence of an acre.console=1 query arg instead of the ‘/acre/preview’ path prefix we used to use.

console.log to Firebug with FirePHP
The switch to header-based logging also now enables the use of the FirePHP Firefox extension with Acre apps. If you can live with the PHP formatting, this is a great debugging tool as you’ll see all of your console.log statements for every single Acre request… including AJAX requests and POSTs.

New App Editor Design
To accommodate all of these new features (and a few more coming soon), we’ve redesigned the App Editor. The redesign better surfaces the most commonly-used editor functions as buttons while still showing more code. We also got feedback that having the App and File menus next to each other was confusing, so the App menu is now separate and triggered by clicking on the app name in the header. There are many more improvements too, like IE7 and IE8 support… so just check it out for yourself.

New Code Editor
We’ve also upgraded the version of the CodeMirror library that we use for editing. In addition to fixing a number of bugs, the current line will now be remembered between file switches. Indentation is also no longer forced. So:

• Tab key now shifts a line to the right rather than re-setting indentation
• Highlighting a section and pressing tab shifts the whole section
• To re-indent a selection, press shift-cmd-i on a Mac and shift-ctrl- i on a PC.

Acre API updates

• acre.freebase.img_thumb(id, height, width, mode, errorid) has two new arguments: mode and errorid. Mode takes any of ‘fit’, ‘fill’, ‘fillcrop’ and ‘fillcropmid’, while ‘errorid’ is the ID of the placeholder image to use if an image is not found.
• acre.form_url(base_url, dict). This is a new helper API that will take a base URL and append a serialized Javascript dictionary as a query string to it. This is basically the reverse of acre.environ.params.

Acre API Reference & Code Search
We’re experimenting with a new API browser (much more current and easy to browse, but still lacking some detail), as well as making all Acre scripts searchable so you can find example uses of different APIs. You can try out both from the Help menu in App Editor or directly at
http://api.freebaseapps.com/ and http://codesearch.freebaseapps.com/.

Faster JSON parsing
All of our JSON parsing used to be done in Javascript, which for large requests (like, say… a big mqlread :-) could get quite slow. In this release, we’ve switched to a native Java JSON parser which is much faster.