Recent blog entries for chromatic

The Perl 6 design team met by phone on 30 July 2008. Allison, Jerry, Will, Jesse, and chromatic attended.

Allison:

• OSCON was a success
• now I'm completely free from all responsibilities until at least December
• launched the PIR PDD
• now working on the last few failing language tests before I merge back the concurrency branch
• also spent some time talking to PhD advisors at Cambridge this week
• might start fall 2009, if I can get funding and such
• leaving for London on Saturday
• speaking at the BBC on Tuesday
• may miss next week's meetings

c:

• was very busy with OSCON
• trying to apply a few patches
• trying to get other people to the place where they can help with that
• mostly I need tasks I can do in 30 minutes when I can grab an hour or two
• fixed some small stuff at the hackathon

Allison:

• I need to split up the MMD PDD into bite-sized tasks

Jerry:

• OSCON was fabulous
• our talk was successful
• good feedback and enthusiasm
• the lighting talks with Jeff were successful
• most of my hacking was on mod_parrot
• trying to refactor its configure system into something more robust and portable
• it's just about working in the same state it was before
• hope to reach that point this afternoon
• met with some folks at Microsoft's open source lab on Monday
• promising beginning to a relationship between them and the Parrot Foundation
• the hackathon was productive as well

Jesse:

• I talked to some Sun people about what makes CPAN tick

Will:

• mostly absent the past couple of weeks
• trying to resurrect Tcl, which was untouched for some time
• making progress there on spec tests
• identifying what's left to do

c:

• what's our roadmap look like for the rest of the year?

Jesse:

• I asked Patrick about that
• he said there's a good chance to get that from the hackathon
• if not, he said I could get his attention at YAPC and get his milestones to break down soon

Jerry:

• he spoke on IRC about that today
• he wants to break it down into better granularity and put dependencies between tasks
• have a feeling it'll happen at YAPC::EU
• Jonathan is keen to get together with Allison and Patrick there to discuss related designs

c:

• I've noticed projects in Parrot like Reini's attempt to get installation and loading sorted out
• seems like time to get those sorted out

Jerry:

• it's difficult to have willing volunteers and not get them the right guidance
• and to make "eventually" now

c:

• thinking we should ask Reini for a list of concerns, needs, and use cases
• then try to make a design that addresses more of those

Jerry:

• especially as he's about to leave on a two-week business trip
• sounds like a PDD to me

Allison:

• sounds like several

Jerry:

• library loading, runtime components

c:

• does he have commit access?

Jerry:

• we can encourage him to apply

c:

• thinking about bindings to Prophet
• probably client-side

Jerry:

• we need sockets

Will:

• we have experimental sockets
• we need to make tickets or mark up PDDs for unimplemented things
• I did some of that with PDD 19 today
• stuff's deprecated and only exists in the PDD
• just to warn people who are going to write code that docs don't match reality

Jerry:

• that merits a CAGE task for reviewing the PDDs
• we could spread that out among many people

Will:

• I'll send an email

c:

• and post it on parrotblog

Will:

• no one reads those

c:

• they won't if we don't publish them

Jesse:

• I see them posted places like Reddit fairly often

Allison:

• probably will roll that into parrot.org eventually

(Originally posted to p5p and inspired by Re: forcing older perl to use an older version of cpan module?, by Nicholas.)

Maybe it's possible to solve [the problem of businesses having trouble upgrading Perl installations and the lack of volunteers supporting Perl 5] by giving up the polite fiction that the feature-based release process works.

If you (not Nicholas) think it works, answer me this question: with eight months of bug fixes and updates and patches to bleadperl, will 5.10 magically become "stable" and not "testing" only when someone works out the proper given/when semantics? As far as I can tell, that's the main work standing in the way of a 5.10.1 release, at which point people start to believe that they should use 5.10 instead of 5.8.

Answer me this: if you believe that the best upgrades are infrequent upgrades, why do you update your source tree right before you make a new change?

Answer me this: what happens when someone asks "Why is argument unpacking so much slower in 5.10.0 than in 5.8.8?" and no one can point to a vendor backporting patches aggressively and the best answer there is is "There's a fix in what will become 5.10.1... eventually... someday"? (Guess how the backport avalanche begins.)

Most of all, answer me this: why does the oft-proposed solution to the problem that "It's difficult to upgrade Perl and its ecosystem" include slowing things down and throwing money and time and talent at boring, manual slog-work of verifying that mountains of changes in once-in-a-blue-moon big bang big thud releases work? Why not make upgrading Perl so simple and easy that it's boring and easy to automate because it's frequent, reliable, and repeatable?

In return, I'll answer two questions premptively, for free.

1) It's possible because the Linux kernel does it -- and if you want a project that's larger and older than Perl 5, has orders of magnitude more changes, and is very possibly more widespread, there's your example.

2) The reason to upgrade every month or two months or three months is because bleadperl is better than it was three or two or one month(s) ago. (If it's not, that's a different problem, and we should probably stop developing Perl if we can't fix that.)

The Perl 6 design team met by phone on 16 July 2008. Larry, Allison, Jerry, Will, Jesse, and chromatic attended.

Larry:

• switched my laptop to Ubuntu for various reasons, mostly work
• helped me get a runnable Pugs too
• haven't worked much on the standard grammar
• mostly thinking about refactoring the names
• thinking about various other questions people are asking
• just checked in a revamp of the closure semantics
• a little under the weather too

Allison:

• almost entirely absorbed by OSCON this week
• hacking a little bit on the three remaining test failures on the pdd25cx branch
• Patrick gave me some information on the PGE failures
• we'll work on that tonight
• hopefully will merge in the branch finally

Jerry:

• haven't done much this week
• will report for Patrick and Jonathan though
• Patrick's working on lexical issues
• he's much closer to understanding how Parrot can handle them
• he's also working with HLL to get it to work within PCT
• then he can make it work within high-level languages itself
• that'll let them use the same named classes as Parrot's base classes (for example, Integer) instead of funky workarounds
• Jonathan added enum support
• fixed a couple of segfaults
• something like 910 new spectests passing between Parrot 0.6.3 and 0.6.4
• that's a great number
• much of it came from the Summer of Code project for the Perl 6 test suite
• Adrian and Moritz have been a big help there

Patrick: (in a followup)

The primarily reason we want HLL support is to take advantage of HLL type mapping, so that str register-to-PMC conversion automatically gives me back a Perl 6 Str object instead of a Parrot String PMC. Same for int to Int, num to Num, and ResizablePMCArray to List conversions.

c:

• things are pretty slow
• traveling and haven't had much time
• still working with Andrew on the GC; he's making progress
• helping Allison; we're getting close
• will definitely start the string branch very soon
• maybe we'll merge that back in stages
• waiting for some guidance on the closure issue

Will:

• Parrot 0.6.4 came out on Tuesday, thanks to Bernhard
• no major new features, although the NEWS listing has lots of little updates
• hopefully will get the concurrency branch merged by the next release, to make it 0.7
• I hate Subversion
• I wonder when we're going to update RT on perl.org
• and that's it

Jesse:

• talked to Richard last week about other interesting funding sources and corporate support for Perl 6
• he's working on that
• general strategizing on that for pushing implementations forward
• sounds like the hackathon is fairly set post-OSCON?

Allison:

• yes

c:

• do we have a sense of prioritization of features, or is that up to implementors?

Jesse:

• it's been up to implementors so far
• every time I've asked, each one has strong feelings about how to prioritize

c:

• I mention it just because I believe that giving people something productive to experiment with sooner is very useful for marketing

Jesse:

• I've talked to Flavio about this, for example
• wondering if there are subsystems of features that could come before others

Jerry:

• what I really want in Rakudo right now is IO so I can use it for work
• it's limited
• you can't do sockets, basically
• that probably won't be fixed until it's fixed in Parrot
• that's Parrot blocking for Rakudo
• as far as other implementations go, there are early features that all implementations share
• I think Larry's said before that the Synopses deliberately don't look ahead too much
• if you do operators before you do objects, that's a way forward
• although parameterized roles are funded work
• and volunteer work

Larry:

• I do use those in my work

c:

• I don't have a sense of our priorities

Larry:

• Perl 6 will be very useful when it does its Perl 5 subset in a way that's better than Perl 5
• that said, there's a danger when Perl 6 gets used in anger by people who start to rely on transient bugs or workarounds for missing features
• there needs to be expectations management

Jerry:

• possibly the best way to do that is to say "Only use what passes in the spectest suite"

Jesse:

• unless there's a way to disable unstable features outside of devel checkouts

Larry:

• but what does it mean that a feature works?
• closure cloning might almost work, but there are some bugs
• we can put a large warning in the documentation that not all features are stable or final yet
• or run into the problem of installing Cabal features to run Pugs
• some packages are under development
• you're lucky if it all works
• it's easy to get an explosion of things that don't all work with each other
• I don't profess to have the right answer
• I don't think there is a right answer
• mostly I think the order of implementation will tend to iron itself out
• everyone has an itch to have the basic stuff working
• everyone has extra itches which may not correspond to other itches
• they're exercising parts of the design that other people aren't exercising
• I want convergence to happen

Jerry:

• provided you don't go crazy trying to keep it all in mind

Larry:

• there may be a need to name development subset versions
• maybe pre-6.0 isn't a sufficiently rich naming scheme
• at least it's an interesting problem

Jesse:

• it would be interesting to find a name for subsets of what will be Perl 6
• the trivial functional subset of Perl 6 that Audrey made in a week was a working subset of Perl 6

Larry:

• and it did junctions

Jesse:

• I've had multiple implementors prod me about how to name or label these things
• here's a thing we've targeted
• we can talk about it as an entity, even if it's not 6.0.0

Larry:

• there's a sense in which the true name is the list of all of the tests it passes

Jerry:

• Rakudo has a ROADMAP
• Patrick keeps it updated
• but he had some unanswered questions in that thread

Larry:

• there's a long list of things blocking from the Parrot end
• a lot of things are in perpetual redesign

Allison:

• I finished the last Parrot design document
• we need resources to implement them

Jerry:

• there were a lot of early prototypes that made it 60 - 80% of the way
• a lot of these features are waiting for a final thing
• it's no fun coding HLLs for a moving target
• they want to wait for the final version, rather than rewriting it several times

Jesse:

• the Rakudo ROADMAP is just a bullet list
• no subtasks, no estimates of effort
• a list in rough priority order
• it could use a couple of more levels of information

Jerry:

• we could make it a hackathon task

Jesse:

• I'll only be there for a couple of hours

Jerry:

• we could pull in some time earlier in the week
• why don't we call these chunks of features "traits"?
• that's an unused term

The Perl 6 design team met by phone on 09 July 2008. Larry, Allison, Jesse, Jerry, Patrick, and chromatic attended.

Larry:

• standard grammar parses itself 100%
• thinking about refactoring
• how do we distinguish the use of grammars from grammars used as the current language
• refactoring top level namespaces
• whether STD is part of that name
• meddling in everyone else's discussions
• upgraded the spec on transliteration
• in denial about all of the conferences coming up

Allison:

• absorbed by OSCON and travel for the next couple of weeks
• after that, things will pick up
• working on one (hopefully) final patch to the concurrency branch
• I know it'll resolve one failing test
• hope it'll resolve several PGE failures on the branch
• I'm an hour away from committing that (ideal minutes, but probably an actual hour)
• we're talking about having a hackathon the Saturday after OSCON

Jesse:

• is there a venue?

Allison:

• nothing set up yet, but we do have a default

Jesse:

• how many people?

Allison:

• probably about ten
• Portland State University worked out pretty well for us
• I'll look for contact information

Patrick:

• last week, we passed 1365 tests
• now we're passing 1677, 312 more
• from the last Friday of June to the first Friday of July, we had an increase of 500 tests
• almost 50% more than we were

Jesse:

• how many are newly ported tests and how many used to fail?

Patrick:

• probably an equal mix of both
• a big part of that is getting implicit lexicals working, $_, $!, and $/
• implicit method calls too are working now
• lots of refactoring, especially for builtins
• they now seem to work better in the Any class
• removed a lot of inline PIR code and stuff like that
• spent an hour on the phone with the Houston Perl Mongers last night
• they want to work on the test suite as part of their meetings for a while
• I walked them through building Rakudo, checking out the Pugs repo and test suite, explaining how we do testing
• they plan to have meetings where someone gives a topic, presents how they think it should work, and then they review or create tests for that feature
• they'll either move tests from Pugs into spectests, or add tests to spectests
• I'll try to write that into an article for use Perl or other places
• a step by step guide for contributing tests
• haven't started writing my "This Week in Perl 6 Articles" yet
• should have one by this time next week

Jerry:

• haven't been able to produce much code
• participating in code reviews and design discussions
• this is the mid-term evaluation period for Google Summer of Code
• all of the mentors and students have submitted their mid-term evaluations
• it's all gone smoothly
• all three are going ahead
• the Parrot Foundation is in process of taking over the NLNet grant from TPF
• working with the OSU open source labs to host the website
• expect more on that soon

Allison:

• because we need SSL certificates, they were going to set us up with our own virtual machine
• we'll need to purchase the certificate

Jerry:

• and finalize the purchase of parrot.org

c:

• applied some patches, fixed a couple of bugs
• closed some tickets, but a lot more came in
• worked with Andrew some on fixing bugs in his GC
• getting closer to the point of merge
• approved his milestone, setting some priorities for the second half of the work
• probably won't have much time to work in the next week
• thinking about Closures, not sure Parrot's approach is entirely right

Patrick:

• did you see my note about the PGE bugs in the branch?
• I ran PGE in the concurrency branch yesterday, and get an odd result
• if you run it by using prove, it gives errors
• if you run the PIR directly, it works just fine

c:

• I don't think Parrot::Test adds any flags when it invokes Parrot
• but t/harness does

Patrick:

• it's in the operator precedence parser
• you can specify a particular stop token
• nothing uses that anywhere anymore, but the test tests it
• I don't think anything that uses it

Allison:

• does it throw an exception?

Patrick:

• it just stops parsing at that point
• the caller decides what to do
• it's not detecting that token when it's parsing
• I pass a string with a stop token in the middle
• it's supposed to parse only the first part
• but it doesn't stop at the stop token
• it's just a simple comparison of some sort
• seems kinda weird

Allison:

• maybe it is detecting it, but isn't working right
• but if you're not using exceptions, there's no old exception handler lying around
• so it's not that

Patrick:

• I gave instructions on how to reproduce in my message

Allison:

• my current patch probably won't fix that
• it sounds unrelated

Patrick:

• I might put some tracing in and run it through the harness
• figure out why it's failing to detect that stop token
• also a question for Larry
• do we have an official leaning on method fallback?
• are they in or out?

Larry:

• assume they're out for now

Patrick:

• I'm happy
• I don't have to rewrite my dispatcher
• can I start telling people to organize the test suite that way?

Larry:

• sure

Patrick:

• for the eval built-in, can someone take String.eval and have it do what you expect?

Larry:

• yes

Patrick:

• a method that's exported
• great!

Jerry:

• STD.pm parses itself
• that's fantastic
• is the LTM implementable now in Rakudo or elsewhere?

Larry:

• essentially

Jerry:

• is anything blocking that?

Patrick:

• just time
• but I plan to develop that under the grant
• I think it'll be a significant chunk of code and research
• don't want to do a bunch of PGE refactoring until the concurrency branch is back in trunk though

Jerry:

• I want to get that merged as soon as possible
• it won't make a lot of progress until we do
• hope we can get it before OSCON, even though Allison will be busy

Allison:

• there are only three failing tests

Jerry:

• but HLLs don't work

Patrick:

• I don't want to go into OSCON without languages working on trunk

Larry:

• maybe a reverse merge?

Allison:

• we do that regularly

c:

• I just don't want to block everyone from doing anything until the code gets fixed

Patrick:

• just about everyone who can fix it is on this call

Allison:

• we're almost to the point where it's time for HLL implementors to work on it

c:

• seems like we can pick the top three HLLs that need to work (Rakudo, Pheme, Punie)
• make them pass
• give the other HLL implementors a week to fix their stuff
• then do the merge

Patrick:

• just don't want to show off all of this nice stuff at OSCON, then say "Don't download trunk, it doesn't work"
• but show off the new stuff in Rakudo that's not in the new release
• a week makes a big difference, the way things are moving

c:

• that could be 500 tests

Patrick:

• thanks to Larry for jumping in on the design discussions
• hope you're not frustrated

Larry:

• it's all part of the job
• I'd rather see progress than vice versa

[ERROR] [Fri Aug 15 07:46:43 2008] This module requires 'Module::Build' and 'CPANPLUS::Dist::Build' to be installed, but you don't have it! Will fall back to 'CPANPLUS::Dist::MM', but might not be able to install!

FAIL Test-Kwalitee-1.01 i386-netbsd-thread-multi-64int 3.1

I didn't choose my distro's requirements randomly. I chose them because they're necessary to build, test, and install my distro. I'm not sure what this failure report should tell me. It won't work without the requirements I've already indicated are requirements?

Someone should write a CPAN testers client in a language other than Perl so people can use it to report that CPAN distributions tend to fail on boxes without Perl installed.

Jesse admitted (in private, anyhow) that my crazy ideas about free software release cycles and support plans may have some merit. He also asked me to explain them in much more detail than pithy quotes in email and on IRC.

First, I want to assert some axioms. I'm doing this separately so that anything which isn't immediately obvious as true can trigger a lot of comments and discussion separate from my thoughts on release cycles and support. Here goes.

• Small, frequent upgrades are easier to manage than large, infrequent upgrades.
• Development freezes longer than a week don't work.
• You can't make volunteers produce work on defined schedules.
• Feature-based releases expand to fill all available time until someone feels guilty for not finishing the feature on a magical, wish-fulfillment schedule, and hey, has it been a couple of years already?
• A release with even one improvement over the previous release is worth releasing on a time-based schedule.
• (Almost) no one but contributors will test an alpha, beta, gamma, or release candidate in the same way that (almost) no one but contributors wants to run nightly trunk builds.
• The only reliable way to get real feedback from real users is to give them a real release.
• Users may complain that a feature doesn't exist, but at a fraction of the volume than if you remove a feature they thought they might want.
• Handing boring scut-work to volunteers is a great way to reduce their motivation.
• It's possible to release software every month without burning out volunteers, patch managers, pumpkings, release managers, or testers -- and the software can get better every month.
• Revision numbers are meaningless, except insofar as they increase monotonically. Revision identifiers (alpha, beta, release candidate) are evidence of an unhealthy release process.
• Rapid deprecation cycles are okay if you provide migration tools.
• Supporting more than two stable versions of a piece of software purely through volunteer effort is crazy.
• If downstream makes substantial changes to the code or to the release process, downstream has just volunteered to support those changes.
• If your project is under active development, the longer the period between stable releases, the less relevant your project is.
• Users who want critical bug fixes and new features without actually upgrading their software also want magic flying candy-dropping ponies.

I expect there may be a few objections to these sixteen axioms.