Thursday, August 26, 2010

Thrift "Hello World!" using PHP

Apache Thrift lets programs talk amongst themselves. No longer must they be isolated or bigoted towards other languages, nor need they use a cumbersome or limited protocol like HTTP or Memcached. Thrift provides a common ground where php web servers can send information to Java collectors, and Python scripts can request feedback from C++ servers.

This post is part of a series exploring Thrift using PHP, with the eventual goal of incorporating Twitter's FlockDB into a PHP Web front end. Today, I present the simplest use of Thrift possible, aptly named, "Hello World." It depends on your first having installed Thrift. I have tested the following code using Thrift 0.2.0 on a Snow Leopard iMac, and made it available for download from Github.

Thrift file


thrift/helloworld.thrift

  1. service HelloWorld {
  2. oneway void hello(1:string juicy)
  3. }

One service, one method taking a string as its single argument. The oneway modifier means the client will not wait for a response after sending the hello request. A natural consequence of this is the return type must be void for oneway methods. Generate php source from the thrift file:

thrift --gen php:server thrift/helloworld.thrift

Two freshly minted php files should have been deposited in gen-php/helloworld: HelloWorld.php and helloworld_types.php. We haven't defined any types, so can ignore helloworld_types.php. The scaffolding of our new service is all in HelloWorld.php. Looking through the source, some things to notice:

  1. include_once $GLOBALS['THRIFT_ROOT'].'/Thrift.php';
  2. include_once $GLOBALS['THRIFT_ROOT'].'/packages/helloworld/helloworld_types.php';

This version of Thrift maintains a liberal reliance on the global THRIFT_ROOT. When we make the actual client and server for HelloWorld, we will have to set the THRIFT_ROOT global. For convenience, I put the whole thrift-0.2.0 directory in /usr/local/thrift-0.2.0, so my THRIFT_ROOT will be /usr/local/thrift-0.2.0/lib/php/src.

Notice the location of helloworld_types.php. The files I just generated have to be placed in the packages directory:

  1. mkdir /usr/local/thrift-0.2.0/lib/php/src/packages
  2. cp -r gen-php/helloworld/ /usr/local/thrift-0.2.0/lib/php/src/packages/helloworld/

The remainder of HelloWorld.php contains type definitions for our HelloWorld service. The interface HelloWorldIf is used by both the client and the server when communicating to the other. The client code, HelloWorldClient, has been completely generated! There is also a processor the server uses to deal with requests as they come in from a client, HelloWorldProcessor. NOTE: If you did not use the 'server' sub-option when generating the php files, the processor will not have been generated. People construct servers using php so infrequently, this is probably a good corner to cut. However, in our example the processor is required if only to avoid getting tri-lingual.

PHP Client


php/stream-client.php

  1. <?php
  2. $GLOBALS['THRIFT_ROOT'] = '/usr/local/thrift-0.2.0/lib/php/src';
  3. require_once $GLOBALS['THRIFT_ROOT'].'/Thrift.php';
  4. require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
  5. require_once $GLOBALS['THRIFT_ROOT'].'/transport/TPhpStream.php';
  6. require_once $GLOBALS['THRIFT_ROOT'].'/packages/helloworld/HelloWorld.php';
  7. $stdout = new TPhpStream(TPhpStream::MODE_W);
  8. $protocol = new TBinaryProtocol($stdout);
  9. $client = new HelloWorldClient($protocol);
  10. $stdout->open();
  11. $send = implode(' ', array_slice($_SERVER['argv'], 1));
  12. $client->hello($send);

Adjust your THRIFT_ROOT for where you have deposited the Thrift download. Usually you would make a client send information through a socket, and that is very possible. But that requires we have a server listening on that socket, and you have to get a little clever to do that in php. So I propose side stepping the issue and creating a client that communicates with STDOUT ($stdout). The binary protocol comes standard with Thrift, use it ($protocol. And we create the HelloWorldClient around the selected stream/protocol pair ($client). Before using the client, the stream (or socket) must be opened. And the string sent to the server will be all the command line arguments. Try it out:

> php php/stream-client.php 'Hello Thrift!!!'
?hello
      Hello Thrift!!! > 

That's pretty ugly, but at least we can see that the raw binary protocol is working.

PHP Server


php/stream-server.php

  1. <?php
  2. $GLOBALS['THRIFT_ROOT'] = '/usr/local/thrift-0.2.0/lib/php/src';
  3. require_once $GLOBALS['THRIFT_ROOT'].'/Thrift.php';
  4. require_once $GLOBALS['THRIFT_ROOT'].'/protocol/TBinaryProtocol.php';
  5. require_once $GLOBALS['THRIFT_ROOT'].'/transport/TPhpStream.php';
  6. require_once $GLOBALS['THRIFT_ROOT'].'/transport/TBufferedTransport.php';
  7. require_once $GLOBALS['THRIFT_ROOT'].'/packages/helloworld/HelloWorld.php';
  8. class HelloWorldHandler implements HelloWorldIf {
  9. public function hello($juicy) {
  10. $time = time();
  11. echo "$time\t$juicy\n";
  12. }
  13. }
  14. $handler = new HelloWorldHandler();
  15. $processor = new HelloWorldProcessor($handler);
  16. $transport = new TBufferedTransport(new TPhpStream(TPhpStream::MODE_R | TPhpStream::MODE_W));
  17. $protocol = new TBinaryProtocol($transport, true, true);
  18. $transport->open();
  19. $processor->process($protocol, $protocol);
  20. $transport->close();

Adjust the THRIFT_ROOT as needed. On the server side we have to actually implement the HelloWorld service. HelloWorldHandler::hello takes the string given, writing it and a timestamp to STDOUT. The rest of this simple example creates a server reading from STDIN and writing to STDOUT. To test it, we can pipe the output from the stream-client to the stream-server:

> php php/stream-client.php 'Hello World!!!' | php php/stream-server.php
1282863497   Hello World!!!

Better! It has begun. We can create a dirt simple client and server using php and the standard streams. Next you might like to try out the slightly more advanced tutorials included with the Thrift download (thrift-0.2.0/tutorial).

Next post: analysis of Flockdb.thrift

Wednesday, August 25, 2010

Install Thrift 0.2.0

INSTALL THRIFT 0.2.0?!!!

It's true. As of this writing, Thrift is already at version 0.4.0. But FlockDB requires version 0.2.0, and for whatever reason is not forwards compatible. QED, install Thrift 0.2.0. Hopefully this situation will sort out soon.

Boost Install

Thrift requires the boost C++ libraries. This script works for me on OS X. Just know that compiling boost takes a very, very long time:

mkdir -p ~/build
cd ~/build
curl --location --remote-name http://downloads.sourceforge.net/project/boost/boost/1.44.0/boost_1_44_0.tar.gz
tar -xf boost_1_44_0.tar.gz
cd boost_1_44_0
./bootstrap.sh
./bjam
sudo ./bjam install

Thrift Install


cd ~/build
curl --location --remote-name http://apache.cs.utah.edu/incubator/thrift/0.2.0-incubating/thrift-0.2.0-incubating.tar.gz
tar -xf thrift-0.2.0-incubating.tar.gz
cd thrift-0.2.0
./configure
sudo make
sudo make install

Test


/usr/local/bin/thrift
Usage: thrift [options] file
Options:
  -version    Print the compiler version
...

There is more bundled into the Thrift 0.2.0 distribution, but I am not sure it remains relevant. Testing that will have to remain a project for another day, probably involving Scribe.

Mile 27 : FlockDB & Thrift 0.2.0

A journey of a thousand miles begins with a single step. — Lao-tzu

Today I continue to struggle with FlockDB and Thrift
Mile 27: Begin blogging the hard stuff

We at ____ have started assimilating Twitter technologies. It began great with the adoption of Kestrel to solve the ubiquitous queuing problem. This works wonderfully. Use Kestrel, it's great!

Now we have a new generalization from the good Robey of Twitter, FlockDB. These pointy birds squawking across the sky are supposed to solve a less Web-unanimous problem, how to scale a massive graph of social connections. They are also the cause of a great deal of death by a thousand pecks to yours truly.

As I am pretty frustrated with the product, and to be fair this is very bleeding-edge stuff, I've decided to blog my progress that it might help the community. But before getting to the promise land of FlockDB, I need to take a detour through Thrift-land.