There are people who you meet and you say to yourself, “how can this guy be so good at so many things?” Steve Vinoski is one of those people. He has a wonderful interview, recorded at Erlang Factory, where he discusses the key concepts of RPC frameworks such as CORBA, and how that compares to REST, and how Erlang fits into both worlds. Interview is here. Steve is the author and maintainer of Yaws, the Erlang web server that makes apache cry like a baby.
PostgreSQL Erlang client library epgsql supports asynchronous messages from LISTEN/NOTIFY events
If you use PostgreSQL with Erlang, you’re probably already familiar with the epgsql client library.
Something that slipped my attention was the addition of LISTEN/NOTIFY support for the Erlang PostgreSQL driver, epgsql. This is an incredibly interesting feature that allows you to wake up any client that has executed a LISTEN
Asynchronous Messages
PostgreSQL may deliver two types of asynchronous message: “notices” in response
to notice and warning messages generated by the server, and “notifications” which
are generated by the LISTEN/NOTIFY mechanism.Passing the {async, Pid} option to pgsql:connect will result in these async
messages being sent to the specified process, otherwise they will be dropped.
I’m curious if there are people using this functionality out there (perhaps Zotonic?)
New Erlang job board – totally-erlang.com
Marc Worrell over at Zotonic says
“We are looking for an Erlang programmer and found that it is quite difficult to find someone. We also found that the available positions are spread out over many websites. Which makes it hard to find interesting positions for those that are looking for Erlang work.”
So Marc put together http://totally-erlang.com/ to quickly (and freely) publish any Erlang positions. If you know of positions, pass them on.
erldocs.com updated with R14A support, mochiweb, and available for your own projects
Dale Harvey just recently updated erldocs.com with R14A support. If you are doing any erlang development, or even looking to just learn a bit about erlang, I highly recommend using erldocs over the official documentation. The ease of use just makes looking up a module, function, or argument so much easier. And the search is real-time so you can basically use it as a discovery tool. For instance, did you know Erlang has an IDL compiler? or a tftp server? What’s your favorite heretofore unknown Erlang standard library ‘gem’?
Additionally, http://github.com/daleharvey/erldocs.com has instructions for how to use with your own libraries and now includes support for mochiweb…
Spanish Language Erlang List
If you’re looking for Spanish language group to discuss Erlang, Mariano Guerra recently started “ErlAr“, a mailing list for all things Erlang, but in Castellano.
Mariano is the author of Efene, a Javascript-syntax language that compiles to Erlang bytecode. I’m planning an interview with Mariano on Efene soon.
There is a small but growing group of Erlangers in Latin America; here’s hoping they continue to grow, but if you speak Spanish, or know someone who does, please join or pass along the invite.
RabbitMQ acquired by SpringSource/VMWare
Erlang-based RabbitMQ was acquired today by SpringSource, which is owned by VMWare. Congrats to Alexis and the team. As SpringSource provides commercial support for ActiveMQ it’s obviously a perfect fit, and it gives commercial players who want to try out Erlang no excuses with regard to support – RabbitMQ was already faster (from my experience), so there were no other excuses left.
There are a number of ways VMWare could use Rabbit on the server side, so it’s a good fit there as well.
Additionally, Redis is at VMWare, and it’s interesting that two of the ‘hottest’ queueing/caching infrastructure technologies are now in the same “home”, or at least neighbors. Redis has been adding a number of pub/sub features in the latest release and has the freedom to not be bound to silly working-group hydras like the AMQP spec. Perhaps RabbitMQ will be able to broaden the offering because they’re inside a bigger entity and not as defined as simply an AMQP provider.
Simple Chart Of Embeddable Scripting Options In Erlang
In preparation for my upcoming talk this week at Erlang Factory San Francisco, focused on Lua integration in Erlang, I’m reviewing Lua, Javascript, and Reia, and I came up with this very simple decision tree: download
Please send any thoughts, or additional languages I should cover. I left out languages that aren’t truly ‘embeddable’, though technically you can access anything via a Port, or BERT-RPC, Protocol Buffers, HTTP, sockets, etc. I wanted to capture the projects that are genuine alternatives to Lua, which is the focus of my talk.
For Lua, there are two projects – erl-lua and erlua. The first, erl-lua is a linked-in driver, originally by Ray Morgan, updated by Darrik Mazey, available on github. erlua is available on gitorious, and is an Erlang Port. erlang_js is written by Kevin Smith of basho. Reia is well known to Erlang Inside readers.
Looking forward to the conference this week!
Zotonic rethinks the CMS with Erlang
I took a minute from getting ready for my talk at Erlang Factory (and working on the day job) to talk with Marc Worrell, Lead Architect of Zotonic – a new Content Management System written entirely in Erlang. This is a brief overview of the system focused on the ‘why’ – the technical details are well covered on their site, but I wanted to understand the motivations behind a company taking on a project like this. Marc also runs his own programming consultancy, WhatWebWhat.
I’m excited to experiment with Zotonic, but I frankly don’t have a downloaded copy of Postgres lying around. I’m planning on getting that installed in order to try it out and will report back.
Here is the interview, as usual, emphasis mine.
1. Hi Marc, can you tell me the impetus for Zotonic?
I used to be the CTO at Mediamatic. Mediamatic built and maintains the community management system Anymeta. Anymeta grew from a simple library to a full fledged community management system. For that we added more and more interactivity and connectivity. Which brought us in a head-on collision with PHP.
PHP is far from stellar in handling many small requests, can’t keep connections open and gets really slow when your codebase grows. Though we need open connections to user agents, XMPP connectivity and quick turnaround on Ajax requests. All are challenging for PHP. So we ended up adding more and more system services to make PHP perform and have the system support our requirements.
Growing a site with PHP starts with opcode caches, memcached, a modified memcached with dependency checking, extra servers and then more servers. Finally adding XMPP server and client processes written in Python. A whole lot of extra software.
And then we were still rewriting PHP tables into code so that it performed better on application load.
Frustration, especially considering my background with building embedded development platforms. Embedded systems and the programming language C show what you can do with the available CPU power.
After I left Mediamatic I started ask myself how to do this better, how to support those multi-core cpus, how to use those gigabytes of memory and how to have all the connectivity a modern website (information hub) needs.
I checked out Ruby on Rails, I checked Django and some other frameworks. They promise a lot. They fall short when you benchmark their efficiency on modern cpus and server systems.
That is when I rediscovered Erlang. The book from Joe really brought me to the realisation that Erlang could solve most, if not all, the problems I was encountering. And his approach to failure was nothing short of a revelation to me.
This lay the seed for Zotonic, as there was nothing else.
2. The Zotonic team seems to have a lot of experience, what brought you together, and what do you do now? Is Zotonic your full-time venture or are you doing something else?
We all know each other from Mediamatic. Tim and Arjan still work for Mediamatic, Peet is a former colleague and now works for Mirabeau. I am currently a freelancer. Zotonic is my tool to deliver great looking web sites that perform well. You could say that it is my full time occupation. Tim and Arjan are part time working on Zotonic.
We get more and more project requests, so we are happy that we did make the right decision to start Zotonic and make it Open Source as well.
3. Why Erlang? How does it make a CMS easier if most will be non-distributed/one-box systems, and Erlang isn’t known for Raw I/O performance? How are you taking advantage of Erlang’s sweet-spot of easy distributed message passing?
We are now using Erlang’s excellent failure handling and multiprocessing capabilities. I think that the failure handling and with it the possibility to make robust systems is one of the often overlooked features of Erlang, when not the most important.
We don’t use the distributed nature of Erlang, yet. But that might change quite soon.
That text processing is essential for web apps is a misconception. Most web apps only parse requests, fetch some data, concatenate a whole lot of text chunks and then output them over a socket back to the user agent. You don’t need text processing for that. In fact on low end machines Erlang is quite capable of saturating a gigabit connection. Informal benchmarks gave more than 6000 page requests per second (famous helloworld templates) on a low end server. That should be enough.
To achieve this performance we made some rather big changes to Webmachine. Basho will be looking into incorporating those changes into the mainstream Webmachine code.
And programming in a functional language is fun
4. NoSQL support? is it important for a CMS? Are there plans to support Riak or Couch or MongoDB?
NoSQL is really the story of the day. A lot of people see it as a means to an end. Some think you need it for performance. That is not true. An external party did some benchmarking on Zotonic and expected the database to be the main bottle neck. It was not, by far.
As I often say, the best way to achieve high performance is to not hit the database. Add some caching and the current RDBMS products can bring you a long way, especially on modern hardware. When you don’t need millions of objects and 99.99% uptime guarantees then I suggest that you just use a RDBMS. Your data will be safe and the administrators of your customer will be happy that they can access the data easily.
Having said that, we are looking into Riak and some other distributed stores. We have some projects at hand that require a scale and uptime that will make us move to a distributed system with elegant failure handling. Only with these requirements a distributed store becomes important.
5. How do you guys use Zotonic yourselves? From an end-user perspective, how does it differ from WordPress or Drupal?
A first impression is important. And the first impression of Zotonic is its speed. I had people asking things like “stop… did I really see five pages just now?” They are used to the slowness of WordPress and Drupal, at least when not using Google, Yahoo or one of the other big sites.
Over time there will be another big difference: realtime. We are in the process of adding more and more realtime features. They enable realtime communicationa and federation of data. It doesn’t matter where your data (profile, blog etc.) is, it matters that it is up to date and can be published where you want it.
For ourselves, we are moving all our sites, blogs, and whatnot to Zotonic. It is more fun in Zotonic and we are eager to use the PubSub capabilities Arjan just added.
6. From the IT-guy perspective, why should he roll out Zotonic?
You need less hardware for more visitors.
Let’s give an example.
We made the site for the New Island Festival, a cultural festival in New York. At some point we were seeing sustained traffic of 2 mbit. What made that traffic impressive was that the site was hosted on a small VPS and only using 5-10% cpu. We have seen other requirements for PHP sites
7. What are the largest deployments so far?
The New Island Festival was a nice test bed for the number of visitors. A bit bigger, content-wise, site we just finished is the site for the Mediafonds in the Netherlands. This site has around 25000 pages.
We are now working on a project with a projected page count of around a million and later we will have a project that requires a ten fold higher page count.
8. Roadmap – what are your plans for the future of Zotonic?
We have now scheduled or are planning:
- redesign of the admin environment.
- redesign of the access permission system to better support user generated content.
- extended support of XMPP PubSub, also sharing images and other metadata.
- realtime tools for server monitoring.
- support for distributed Zotonic nodes (still with a central store).
- support for a distributed, fault tolerant data and file store.And many smaller changes that will make the system better for building the websites our customers and programmers love.
9. MySQL Support? Is it planned?
No, we don’t plan MySQL support. Though it should be doable with the new binary driver. Right now we are focusing on the distributed version with a k/v store.
edit 21-jul: i took out the word ‘destroys WordPress’ from the title; upon reflection, realizing my Erlang zeal got the better of me..
Erlang Factory SF Bay 2010 – Francesco Cesarini on the Conference and Old-School vs New-School Erlangers
The Erlang Factory’s 2010 conference is March 25th and 26th in San Francisco, with the university three days before, starting the 22nd. The conference is at the Hilton San Francisco Airport, with three tracks, each on a different theme. The number of tracks gives attendees an unusually broad set of choices for talks. When attending conferences I use the times when one or both talks are uninteresting to check email and generally hang out with people. Occasionally there are two talks I want to see at once so I’ll pick one and then duck out to catch the end of the other. Joe Armstrong and Nick Gerakine’s simultaneous talks on Day 2, for example. However, in this case there will at times be three interesting subjects running concurrently, which is only appropriate for a conference on Erlang.
The conference is USD $500 until TOMORROW January 31st, then the price goes up to USD $700.
(However, if you can’t make it to the Bay Area, the Erlang Factory London is June 10th and 11th with the university the three days before.)
We’re going to be doing a series of posts on the event with interviews with the keynote speakers, starting with conference organizer Francesco Cesarini, founder of Erlang Solutions Ltd. Both conferences and universities were smashing successes last year so I expect that they will be well attended and worth the price of admission. I’m personally making it back to the states as well to go for the first time.
Erlang Inside: How many attendees will be at the SF Bay Event?
Francesco C: Erlang Factory is becoming a very popular event. We are expecting an increase in participants, somewhere between 150 – 200 delegates. We have seen a growing trend, irrespective of the state of the economy.
EI: What is the experience level of the average developer?
FC: It varies. Last year, we had students who never used Erlang in a commercial project but were interested in learning more. They were interacting with people who have been working with Erlang full time for 15 years. Even with this huge gap, you did not notice any divide. That’s the beauty of our conferences; Erlang enthusiasts from architects to newbies have an opportunity to exchange knowledge, ideas, experience and have a great time together. Of those attending, about 50% came from the west coast, 17% from the east coast, 15% from the mid-west and 18% from the rest of the world, mainly Europe, but also from Canada, Central and South America.
EI: What percentage of people go to the university and the conference?
FC: About 20% attend the Erlang University. The value of attending the courses in this way is that they are condensed versions of the 5-day courses and allow delegates to attend the training as well as the conference. Furthermore, having the training together with the conference provides the opportunity to discussions and networking, multiplying the value many times. New for this year is a course by Basho Technologies‘ very own Kevin Smith on Web development in Erlang. This course will run alongside the OTP Express course, taught by Erlang Solutions’ training manager Henry Nystrom, Erlang Express, taught by Erlang Programming O’Reilly Author Simon Thompson and Quick Check Express, taught by Prof. John Hughes and Prof. Thomas Arts.
EI: How has the attendance and the demographic of the group changed in the past few years? Last year people talked about ‘old school’ and ’new school’ Erlang crowd. Do you see this as a genuine dichotomy and if so what does it mean for the community?
FC: It felt very strange being referred to the Old Guard last year. But during the first day of the conference, I heard newcomers comment on approachable and friendly everyone was. As one of the speakers put it, the Erlang community seems to lack Prima Donnas…. Which is an advantage. Gone are the days when you knew everyone on the erlang-questions mailing list personally, but the community still feels small and welcoming. Where else do you see inventors of the language helping newbies on mailing lists? Those who have been around a while get excited at all the new projects and companies adopting Erlang, and will help out to ensure it happens. This is what we have all collectively been working for, and seeing presentations from facebook, E*Trade, Yahoo! or SAP (To mention but a few of the success stories) makes us all very proud.
EI: Do you see a shift from Europe toward North America? or Asia?
FC: The community seems to be growing the fastest in North America, Eastern Europe and China, but have no hard facts to back this up other than tweets, blog posts and websites. In 2004, it was easy to derive by examining the email addresses of the erlang-questions mailing list subscribers. Today, the majority of subscribers seems to be using a gmail account. We are aggregating concrete data from Erlang related websites, book sales and other sources which we hope to present in San Francisco.
EI: In training classes, do you see the push for Erlang coming from inside larger corporations, startups, consultants? Or is it more driven by technical needs – multicore, distributed systems, etc.?
FC: It is very varied. In the US, we have a lot of private individuals attending the training courses and the conferences. In Europe, it is mainly companies; everything from one man band consultancies, small start-ups to multinational corporations. This difference is cultural, where the responsibility is moved from the employee to the employer. These companies, in turn, are using Erlang to solve problems related to distribution, concurrency, reliability and scaling on multicore. Five to ten years ago, Erlang was primarily used within Telecoms. Today, you will find it in finance, banking, messaging, web development, as well as in cloud computing backbones. Another emerging area covers test automation and property based testing.
EI: What can the average developer who is not using Erlang expect to get out of the conference and university?
FC: The Erlang Factory is a conference not only for Erlang Developers but also for non technical people, including managers, testers and system architects. We advocate the right tool for the right job, so expect to meet Ruby and Haskell enthusiasts alongside java and python developers, all there to learn more. If you are thinking of introducing Erlang in your organization or just want to learn more about it, this is the event to be. This is the major difference with the User Conferences, which are primarily aimed at developers.
EI: What about the Erlang developer using it on a daily basis?
FC: Those using Erlang on a daily basis will be able to get the latest news directly from the horses’ mouth, meeting members of the community they have previously only met in ascii format. With the ratio of four delegates to every speaker, the Erlang factory gives everyone the opportunity to network, provide feedback on tools and libraries, and discuss ideas, projects and solutions. Many have even found new jobs this way.
EI: Any other thoughts on the conference in SF or London for people thinking about going?
FC: Don’t hesitate to try it out one year and we promise you will come back for more, maybe not only as a delegate, but also as a Speaker. With Joe Armstrong and Steve Vinoski in San Francisco, and Robert Virding and Martin Odersky in London being just some of the 35+ speakers at each event who have confirmed, the Erlang Factory has now become the largest Erlang conference around. We are also working hard to reduce costs so as to enable more people to attend. This is, something which in San Francisco has been reflected not only in reduced Early Bird rates, but also in the conference hotel room rate, which we managed to negotiate down by $70 per night compared to last year.
Remember tomorrow is the last day to get signed up for $500…
Mustache – GitHub’s framework agnostic templating ported to Erlang
Mustache is an Erlang port of the Ruby framework of the same name, originally written by Chris Wanstrath, one of the founders of GitHub. A powerful feature of this type of framework is the ability to write a ‘template’ that you could use in a Rails application and then reuse in an Erlang application with no changes to that file. The ‘view’, which in MVC terms is more like the ‘controller’ would have to be re-written, but this would greatly simplify the work of porting an application, OR simply running a part of application in a different framework for performance reasons.
Chris calls Mustache “a replacement for your views”. Instead of views consisting of ERB or HAML with random helpers and arbitrary logic, your views are broken into two parts: a Ruby class (or Erlang module) and an HTML template.
They call the Ruby/Erlang part the “view” and the HTML template the “template.”
All your logic, decisions, and code is contained in your view. All your markup is contained in your template. The template does nothing but reference methods in your view.
This strict separation makes it easier to write clean templates, easier to test your views, and more fun to work on your app’s front end.
Check it out!