Out of curiosity, I was looking how a browser interacts with the Google Instant backend. While looking at the HTTP exchanges via Firebug, I first asked myself why they’re encoding HTML and JS with \xYY escape sequences, then why the very same JS functions are sent back and forth on every request, and later I stumbled upon the google.com/s?q=QUERY JSONp service.

Give it a query, and it’ll return the suggested related phrases that are used to build the menu under the search input while using suggestions and/or instant (didn’t dig too much into all the other parameters).

Anyway, what’s interesting is that, of course, the suggestions are customized on a per-country basis. To show the differences explicitly, let’s ask the service the simplest query possible, a:

For Italy you’ll get:

$ curl http://www.google.it/s?q=a
window.google.ac.h(["a",[["ansa","","0"],
["alice","","1"],["alitalia","","2"],["alice mail","","3"],
["apple","","4"],["agenzia delle entrate","","5"],
["audi","","6"],["aci","","7"],["autoscout","","8"],
["atm","","9"]],"","","","","",{}])

hum, let’s scrap the JSONp and parameters out:

$ curl -s http://www.google.it/s?q=a | ruby -rjson -ne 'puts JSON($_[19..-2])[1].map(&:first).join(", ")'            
ansa, alice, alitalia, alice mail, apple, agenzia delle entrate, audi, aci, autoscout, atm

For the US you’ll get:

amazon, aol, att, apple, american airlines, abc, ask.com, amtrak, addicting games, aim

UK:

argos, amazon, asda, asos, autotrader, aa route planner, aol, apple, amazon uk, aqa

Ireland:

aer lingus, aib, argos, amazon.co.uk, argos.ie, asos, aa route planner, amazon, aldi, aib internet banking

Lastly, because I’ve been there lately and it has been a profound experience, Cuba:

asus, antonio maceo, amor, amigos, ain, antivirus, avira, alba, aduana, as

I’m sure @nhaima is smiling while seeing these words, because hell yeah, over there they really google antivirus software (avira is one of them) a lot because it’s a world without the Internet, thus without free software: you’re condemned in using Windows stuff, and you take what you pay for. Antonio Maceo has been an hero of the 19th century revolution, and it’s in the heart of Cuban people. Amor, Amigos! :-)

Anyway, looks like that simple queries like this really give an insight on what a population thinks and/or needs, because they’re surely generated by the search trends, thus are the “most searched words”. Am I discovering hot water? Maybe, but it was funny to rediscover it. Just make sure not to hammer the /s service with too many requests, because they’ll anyway be handled by the same cluster of machines, thus you’ll be banned early (I’ve been :-p).

On July 22nd 2010, Mikamai hosted a Ruby Social Club in Milan , where nearly 50 people attended watching five speeches about Ruby, Web development and Startups. I was glad to be one of the speakers, and I presented a set of Rails plugins we spinned off from our latest (and greatest) project: Panmind (read more on the about page ) and released as Open Source on GitHub .

The keynote is split in two parts: the first one explains why you should follow the sane software engineering principle of writing modular and interest-separated code and then how you could (and should) extract it from your Rails application by decoupling configuration and then prepare for the Open Source release, by writing documentation AND presenting to a Ruby event so, hopefully, someone else will write unit tests! :-)

We released an SSL helper plugin that implements filters (like Rails’ ssl_requirement) but also named route helpers: no more <%= url_for :protocol => 'https' %>! You’ll have something like plain_root_url and ssl_login_url - like they were built into the framework.

Then, a Google Analytics ultra-simple plugin, with <noscript> support, a couple of test helpers and an embryo of a JS Analytics framework - hopefully it’ll evolve into a complete jQuery plugin. Then, a ReCaptcha interface, with AJAX validation support and eventually a Zendesk interface for Rails.

We released also more code on Panmind’s GitHub account , including the nifty AJAX Navigation Framework that implements all the boilerplate code for the ultra-fast AJAX navigation of panmind contents and projects .

The keynote follows, you can download it in PDF (no exploits, I swear! ) from this link or view/comment it on slideshare here .

Final words: check out mikamai blog post on the Ruby Social Club to read the other keynotes (I will, hopefully, update this post with sum-ups of them when time permits :-)) and say hello on twitter or on GitHub if you’re interested in contributing our open source projects or you want to work with us .

PDF version

As most of you already know, there are two open, critical vulnerabilities in iPhone OS versions from 3.x up. The first one resides in the Compact Font Format component of the PDF renderer and the second one an error in the kernel, allowing attackers to bypass the sandbox (SeatBelt) inside which applications are run on the iPhone.

The two vulnerabilities were discovered by @comex , @chpwn and other people.

Only few weeks later the .lnk design flaw on windows (guys, you’re using LoadLibraryW to load a damn icon!), these iPhone OS vulnerabilities are even more interesting, because of the way the release is being handled by the community and the vendor.

I spent 3 hours last night trying to find detalied information about the bug, and except confused (and propagandistic) blog posts the only bit of information is in this tweet , and in the actual pdf exploit running on jailbreakme.com . Where are the security lists posts? Where is the CVE? Even the CERT still doesn’t say anything about this vulnerability.

There’s something terribly wrong going on: the cat-and-mouse-game that is making the iphone-dev team researchers not disclose any of the vulnerabilities they find has become very dangerous for end users: an exploit that allows remote code execution and jail escape without no interaction whatsoever by the user, carried via something that’s used to consider “safe” (a PDF file) is what is called a critical hole; while the exploit that uses it is called a 0-day. It’s the first time in my life I see a 0-day packaged and distributed explicitly via a web site.

Anyway, the dev-team researchers did not have any other choice: if they had communicated with Apple prior to public disclosure, we wouldn’t have had a so easy jailbreak vector; OTOH now we have vulnerable phones and pads that can be very easily exploited by mailcious parties. It’s also funny that in order to be warned when a PDF is about to be loaded thus mitigating the risk, you should jailbreak your device and install the PDF Loading Warner afterhand.

My stand on this is that the real problem is Apple itself: they’ve crated a walled garden , outside any legislation, where they’re the absolute god and give and take whatever they want. It’s not gonna work forever. I really hope that people will understand think that it’s not the hackers’ fault, rather it’s the totalitarian companies’ fault, for not giving us control over the devices we buy from them. Hackers are only trying to liberate them, and it’s fair use under the DMCA, after all .

UPDATE 2010-10-05: I’ve posted a summary of this bug on the full-disclosure mailing list – you know, if it’s not on FD no one would think about it :-).

In a nutshell, it adds support for unmarshaling 1.9 strings, and implements the last missing type (TYPE_LINK) that was missing from the code. Tests still lack, can someone help ? :-)

Added TYPE_LINK, needed because of how ruby 1.9 marshals strings.

In 1.9, Ruby marshals the string encoding in the binary output, and
uses an Ivar construct (TYPE_IVAR) to wrap the string and adds an
"encoding" instance variable (notice: without a leading @) whose
value is the encoding itself.

While the Ivar code worked correctly, the values of the encodings
are actually *strings*, that are being reused via the TYPE_LINK
construct, that wasn't implemented.

So, the get() and put() primitives are being used to store not
only tuples {id, sym} for symbols, but now store either

  {{symbol, ID}, sym}

  OR

  {{value,  ID}, val}

for the other types that use TYPE_LINK.

By reading the ruby marshal.c source code, it looks like that MANY
data types save their values in the arg->data hashtable, but by
inspecting the binary marshal output of, e.g, an array of floats,
links aren't used.

Thus, in this unmarshaler, links are considered, for now, only for
strings and regexes.

Fork me on GitHub: http://github.com/vjt/erlang-ruby-marshal

If your CouchDB 0.11 gives you the “Invalid UTF-8 JSON” error on every POST or PUT you issue to it, make sure that in your $prefix/usr/lib/couchdb/erlang/lib there aren’t leftovers from previous installations.

On our dev server, I found there two directories (“couch-0.10” and “mochiweb-r97”) from the old 0.10 setup that were causing this issue.

This applies if you upgraded from source, as you’ve probably did, because there aren’t too many packages of CouchDB 0.11 as of April 2010 :-).

Huge thanks to @couchdb for hinting me in the right direction after reading a report on the dev mailing list but I didn’t want to “remove and reinstall” because I like to understand what’s going on ;-).

Footnote: could this be the end of Hiatus? I hope so ;-p