Take the 2-minute tour ×

The term "Lisp" (or "Lisp-like") is an umbrella for lots of different languages, such as Common Lisp, Scheme, and Arc. There is similar fragmentation in other language communities, like in ML.

However, Ruby and Python have both managed to avoid this fate, where innovation occurred more on the implementation (like PyPy or YARV) instead of making changes to the language itself.

Did the Ruby and Python communities do something special to prevent language fragmentation?

share|improve this question
You say fragmentation like it's a bad thing. –  Sonia Apr 6 '12 at 1:02
@Sonia From a market-share perspective, fragmentation is often a disaster. –  chrisaycock Apr 6 '12 at 1:21
Are languages in competition with each other? –  Barry Brown Apr 6 '12 at 1:26
@Sonia It can be a bad thing. For example, a library written for Python almost certainly doesn't depend on the implementation, whereas a library written for Lisp may not work in Scheme. –  Kris Harper Apr 7 '12 at 0:03
@Barry Brown: Great point! Languages should not be in market competition with each other. But language vendors are and this often influences language design (I do not think this is the case of Ruby, Python, Lisp, ML, though). –  Giorgio May 6 '12 at 15:58
show 3 more comments

8 Answers

up vote 74 down vote accepted

Ruby and Python both have benevolent dictators at their helm. They are languages deeply rooted in pragmatic concerns. Those are probably the most significant factors inhibiting fragmentation. Lisp and ML, on the other hand, are more like "design by committee" languages, conceived in academia, for theoretical purposes.

Lisp was originally designed by John McCarthy as a practical mathematical notation for computer programs. He never implemented it as an actual programming language; the first implementation was developed by Steve Russell, but he was not a benevolent dictator. Over time, many different implementations of Lisp appeared; Common Lisp was an attempt to standardize them.

Lisp is more of a "family" of languages. So is ML, which followed a similar evolutionary path to Lisp.

share|improve this answer
Hmm, I definitely see dictator status among homogeneous language communities like Objective-C (for iOS apps) and Ada (for Defense Department contracts). In these cases, a higher power demanded adherence, which developers followed just to be able sell their warez. But I've never been required to code in Python (hobbyist project) in the same sense that I might be required to code in C# (.Net component). Ie, I could more easily flee Python than, say, C. Without this threat of use it or you won't make sales, how can a dictator hold onto the flock? That might be a separate question though. –  chrisaycock Apr 6 '12 at 0:09
By "benevolent dictator," I mean that all language changes must go through one person having the vision to keep the language pure. People stay with Python for pragmatic reasons; they like the language, and are productive in it. But not everyone and their brother is allowed to fork it, and still call it Python. –  Robert Harvey Apr 6 '12 at 0:11
@HenrikHansen Haskell is a standard, as Robert mentions. So the HUGS compiler must be compatible with GHC since both call themselves "Haskell". The same standards-based protection extends to C and C++, which is why GCC and Visual Studio must be compatible (assuming no use of proprietary extensions). The curiosity is what happened to Lisp, where there is already a standard (Common Lisp) and yet there are many other Lisps. ML has the same issue where there's Standard ML and yet other MLs. –  chrisaycock Apr 6 '12 at 16:57
"Common Lisp was developed to standardize the divergent variants of Lisp which predated it" --en.wikipedia.org/wiki/Common_Lisp. In other words, there was already fragmentation before the standard was developed. –  Robert Harvey Apr 6 '12 at 17:27
I would even say ML and Lisp are not languages as Python and Ruby are. Lisp and ML are more like "concepts", implemented by several different languages. –  Ubiquité May 6 '12 at 16:28
show 6 more comments

One likely factor is simply age. Lisp and ML are a lot older than Python and Ruby:

  • Lisp: 1958

  • ML: 1973

  • Python: 1991

  • Ruby: 1995

Lisp and ML have obviously seen much greater change in hardware capabilities, more trends in computer science, and a great many more Ph.D students looking for something to work on.

share|improve this answer
Possibly, but I don't recall Fortran having this degree of forking. (There was stuff like Fortran D, but most Fortrans have gone through standardization.) I suppose maybe the age of coalescing might be a factor. –  chrisaycock Apr 6 '12 at 22:30
AFAIK, Fortran had a lot of incompatibility and non-standard extensions and different implementations until standards committees gradually hammered it out, probably because it was more widespread than Lisp and ML. –  erjiang May 6 '12 at 16:17
@erjian: FORTRAN had its incompatibilities hammered out because there was a serious incentive to: business use. LISP, mostly used in academics, never had that luxury. I.e. it's not how widespread its use was, but how affluent its users were. –  MSalters May 7 '12 at 14:50
Or, alternately, variants weren't called FORTRAN. BASIC, when it came out, sure looked like a simplified FORTRAN. –  David Thornley May 7 '12 at 16:10
@MSalters Common Lisp was actually a (fairly successful IMO) effort to hammer out the incompatibilities in the various maclisp dialects dictated among other things by business use(and also DARPA wanted all the research labs it funded to be able to share code more easily). Today other than scheme, clojure and common lisp, there are no practical general purpose lisps, and these three are different enough, have very separate communities with separate cultures and history to not count them as dialects of the same language anymore than java and C++ are. –  Pavel Penev May 7 '12 at 18:41
add comment

They're essentially all implementation defined languages

When it is easy to create a new implementation of a language that is largely compatible with existing code, then hackers being hackers, they go ahead and do it. Everyone writes a Lisp implementation at some point. ML compilers are almost mandatory for grad students in language design -- the language is after all famously well documented.

On the other hand we have the ad hoc and implementation-defined languages. Or languages that are just so complex that it is a signficant barrier to ever producing a viable alternative implementation:

  • ruby; perl; python -- all too implementation-defined to produce viable alternatives
  • ghc haskell and erlang -- well defined, but so hard to do anything that competes with ghc (or erlang) that people don't usually bother

This seeming downside -- languages that are too hard to produce viable alternatives to, have the massive upside: scarce developer resources are concentrated on the one true implementation.

As a historical note, several in the Haskell community actively pursued mergers and concentration of dev effort, recognizing that any splintering of the dev community would mean we'd not succeed. GHC was chosen and championed.

share|improve this answer
I would love to know more about the "actively pursued mergers and concentration". –  Sam Tobin-Hochstadt May 7 '12 at 4:20
Fragmentation is natural. Languages like Python and Ruby are anomolies which happened to not fragment in the main, if you don't count variants not used, e.g. ChinesePython, and variants stagnating at an earlier version, e.g. Jython. There's also survivorship bias here, because most languages with a dictator don't become very popular, e.g. Nermerle, Groovy, Beanshell, Boo, in fact there's probably thousands of them. –  Vorg van Geir Sep 11 '12 at 8:54
Even then, Haskell could still be more practical to reach Python/Ruby maturity status. Haskell's cabal is not a fun tool to use and rather easy to break:. Even Yesod acknowledges it: yesodweb.com/blog/2012/04/cabal-meta Python and Ruby are a lot better in regards to package management. –  Shurane Oct 31 '12 at 20:19
@Shurane Python and Ruby don't type check your packages before integration... –  Don Stewart Nov 5 '12 at 18:50
-1: for "ruby; perl; python -- all too implementation-defined to produce viable alternatives" Jython, IronPython, JRuby, IronRuby, PyPy, Stackless prove you wrong as for implementations (and these are just the major ones). Also, CPython is reference implementation, but not the language definition, this is –  vartec Mar 14 '13 at 16:14
add comment

I would say one factor is a defining platform. For Haskell the platform is the Haskell standard and the GHC (I would imagine). For Ruby it was Ruby on Rails that "defined" the Ruby development platform. For C it was Unix.

Compare that to Lisp, where there was no original kick-ass platform that defined what the language was like. If I recall correctly each Lisp machine had slight differences depending on model and manufacturer. Common Lisp was for some reason not defining. Possibly because of too much competition and reluctance to move to another platform.

This is, of course, entirely speculation from my side. The thought came from the comment replies on Harvey's answer. However, it seems that the defining platform comes in many shapes, but the common property seems to be that it is what gains popularity from.

share|improve this answer
I actually like this idea. I can use many forms of Lisp because none of them have a "killer framework", but if I want to use Rails, I must stick with the canonical Ruby. Certainly it's not the only answer, but I do like your hypothesis. –  chrisaycock Apr 6 '12 at 18:38
i would agree on platform part. If you have a single translator capable of running the language - there would not be much of fragmentation. –  c69 Apr 6 '12 at 21:22
Common lisp didn't settle on a single definition early because people had strong opinions about certain things, e.g. hygienic macros. –  Robert Harvey Apr 6 '12 at 21:55
I both agree and disagree with this. I agree because a 'killer framework' patches the core language with valuable functionality, encourages growth, and allows rapid innovation outside of the standard spec. I disagree because, if the framework maintainers aren't very careful, that rapid increase in innovation could lead to a lot of bloat and/or leaky abstractions that could render it unstable. –  Evan Plaice Apr 6 '12 at 23:12
(cont) Frameworks like jQuery that extend a languages core functionality ideally will die off in the future as the most valuable contributions given by those frameworks are standardized and incorporated into the core. IMHO, frameworks tend to die the fastest because developers generally prefer to reduce/eliminate dependencies as a codebase stabilizes. If the language developers want to stay relevant, they will make that process easier by adapting and adopting framework functionality, and encouraging their users to reduce dependencies on 3rd party frameworks. –  Evan Plaice Apr 6 '12 at 23:19
add comment

Don't forget to weigh the culture driving a language's development

I would also weight the fact that development on python/php is actively done in public. You have one group of individuals nailing down a standard specification that is freely available to anybody/everybody.

Much like the W3C does with the HTML/CSS standard. You have a small group of motivated individuals who control the finer details of what the language is designed to accomplish. Everything goes into a clearly defined specification before it's released to the public.

OTOH, languages like LISP are forked behind closed doors by professors or other individuals that genuinely believe that their perspective on the 'best use' of the language is right. They may be simultaneously right and wrong at the same time because some implementations are great at certain things; while none are the best at everything.

That's not necessarily a bad thing because diversity breeds innovation. Languages like LISP are, and will remain great languages for learning and research because they push the boundaries of understanding.

But the qualities that make an environment good for innovation aren't necessarily beneficial for stability; conversely, the qualities that make an environment good for stability aren't necessarily good for creativity.

When development is based on active collaboration, sometimes individuals are forced to concede for the benefit of the greater whole. Bad for research/good for consistency.

The fact is, we're still living in the wild-west of programming language development. The problem of designing the 'ideal language' is so great that, despite monumental efforts, nobody has come close to solving it.

In the research/academia sector, there's still a lot of room for improvement and innovation. In the commercial sector, where there is an exponential growth of software being use in practical applications and the driving force is simplicity and consistency.

Some languages specialize in the former, some specialize in the latter. Those that try to specialize in both usually don't do either very well and die off.

By both, I'm referring to monolithic languages like VB/C#/Java. It's too early to say but I'd like to see what C# and Python look like in 10 years. At the current pace C# is growing functionality and inconsistency at a rate that makes it look pretty grim. Even with great documentation, it's just too much of a pain to remember all the subtle details and quirks included in the language. It's great for a single developer but as soon as you throw in more developers with unique styles, inconsistency in the codebase grows, quality suffers, and nobody wins. I think there's a lot to be learned from the difficulties Perl presents in a production environment.

share|improve this answer
Ladder? Do you mean latter? –  Giorgio May 6 '12 at 19:56
@Giorgio Yes, I hate it when I misspell that. –  Evan Plaice May 7 '12 at 17:29
add comment

I don't think it's correct to say that languages like Python and Ruby aren't fragmented. Already we're starting to see some fragmentation effects. For instance, Python 3 isn't entirely backwards-compatible with Python 2, so both versions need to be maintained and lots of existing code works only with Python 2. There are some Python spinoffs, too, including PyPy.

Another factor is the age of the languages. The ones most subjected to fragmentation are the older languages and are thus subject to pressures of evolution and revision. Lisp was invented several decades ago, so there has been ample time to take some of its ideas and incorporate them into new languages. C is another example of a fragmented language. While C had only one really major revision to the language itself (K&R to ANSI), there have been numerous spinoffs including C++, Not Quite C, and all the others that share a C-like syntax.

Ruby itself is a "fragmentation" (if you will) of previous languages. Since it incorporates ideas from C, Smalltalk, and Perl (among others), it's currently the language doing the fragmenting. I don't see why we might not see further convolution of Ruby with other languages in the future.

share|improve this answer
-1 because: (1) Python 3.x is not fragmentation. It's just the next step in the language evolution; Python 2.x will be dropped entirely in a few years. (2) Other language implementations that are 99% compatible (the 1% being implementation details and mostly rather obscure) and actively refuse taking part in defining the language are not fragmentation. (3) A vastly different language that shares some common ground and is somewhat compatible (C++ to C) is hardly fragmentation. (4) Accepting ideas from existing languages is not fragmentation, it's the only way one designs a language. –  delnan Apr 6 '12 at 10:35
@delnan: Python 2.x will be dropped entirely in a few years? That's a bit of a silly thing to say, when COBOL and Fortran are still around! –  Mason Wheeler Apr 6 '12 at 23:04
@MasonWheeler I'm talking about development. The VCS will still have old code archived, inofficial binary downloads may stay for decades, and some shops may avoid porting. But I expect that some not-too-far-away day, the vast majority of Python programming happens in Python 3. After all, 2.x feature development ceased a while ago (and won't resume unless you brainwash python-dev), bugfix/security updates are not supposed to continue forever, and a significant portion of libraries are ported to Python 3 with most other ones either looking forward to (e.g. Djano) or being unmaintained. –  delnan Apr 6 '12 at 23:22
@MasonWheeler Oh, and as for Fortran and COBOL: Fortran got a new standard revision om 2008, and COBOL got one in 2002 with a handful of technical reports since then. –  delnan Apr 6 '12 at 23:25
@MasonWheeler Did you know that modern COBOL allows for object oriented programming? –  user1249 Apr 8 '12 at 22:50
show 2 more comments

Lisp is fragmented because it is such a powerful model, the most amazing language ever conceived. Most languages today borrow things that were first implemented in Lisp, so in a way you can say that every language is part of this particular fragmentation. Smalltalk was for instance heavily inspired by Lisp, and Ruby is heavily inspired by Smalltalk. JavaScript is Lisp in a Java-disguise, and so on.. It's all connected, and every language inventor selects his favorite pieces from other languages.

Another factor is that Lisp is probably the easiest programming concept to implement - which is why it's done again and again and again.

share|improve this answer
add comment

Lisp-like languages are too basic and theoretical to be changed dramatically. Grammatical changes (I do not mean to just change the names of commands) would just not fit the functional-programming theory behind them.

But the fact that there are languages like lisp shows that "changes" were already made to lisp anyway. In other words, there are languages made by people who were inspired by lisp or it's theory behind it and made a in-a-way similar new language.

There are also a lot of languages inspired by Python. E.g. Julia, CoffeeScript, etc. which would form their own family of whitespace-sensitive object-oriented languages.

I think, fundamental basics of a language like Python will never really change. Python is object-oriented and therefore has similarities to C++ and Java but it's dynamic and therefore also similar to a lot of script languages.

Well who actually cares about languages? What counts is the purpose: French is similar to Latin but girls who understand French are way hotter ;)

share|improve this answer
add comment

Your Answer


