Advogato blog for chromatic

Perl 6 Design Minutes for 30 June 2010

Sat, 3 Jul 2010 09:09:37 GMT

The Perl 6 design team met by phone on 30 June 2010. Allison, Patrick, and chromatic attended.

Allison:

working on Parrot packages for Debian experimental
seems like a good idea to do that before the 2.6 supported release
there was also a request for Rakudo packages
not sure if I'm the best person to do it

Patrick:

I'm sure we should package Rakudo Star

Allison:

Debian had a packager for those, but I haven't looked at the packages
this'd be an early run of what we'll do with Rakudo Star

Patrick:

we're not quite ready for packaging that yet
maybe a couple of weeks
finished the List and Iterator types for the #30 release
adjusted Rakudo's Associative and Positional roles
much cleaner implementation now
that'll require a few small spec changes
redid Rakudo's container types
more robust
preparing for autovivification of hashes and arrays
expect to finish those in the next couple of days
there was no container model previously; the code was consequently crufty
lots of cleanup of incorrect assumptions
Rakudo lists are now properly lazy
comment syntax fixed
ROADMAP updated
fixed the meaning of Nil; it's defined, not undefined
added the sink prefix (?)
fixed setting of $!
started fixing bugs and closing tickets on Monday, did 15 or 20
mostly already fixed in the previous couple of weeks
looking at the implementation of the series operator
spec is self-contradictory or ambiguous or both
waiting for Larry's clarification
fixed a bug in $*ARGFILES
had a nice contribution of that implementation last week
that behavior works on any set of files, not just those on the command line
working on autoviv
have some regex backtracking bugs to fix
will work on closures after that
put together three new YAPC presentations
the Rakudo Star presentation will become a video cast or a blog post or both

worked on a slew of Parrot optimizations for Rakudo
have a few more to go
might have to create a Rakudo branch temporarily
will try to help merge the new GC
working on a metamodel for Parrot objects, informed by Perl 6 and Moose

Modern Perl: The (Draft) Book

Tue, 29 Jun 2010 00:11:27 GMT

This took longer than I expected, but the draft of the Modern Perl book is available for review. I'm especially interested in hearing from people who don't consider themselves expert Perl 5 programmers. The goal of the book is to explain how Perl 5 works (and how to write Perl 5 effectively) to help novices become adepts.

Perl 6 Design Minutes for 16 June 2010

Sat, 26 Jun 2010 17:09:43 GMT

The Perl 6 design team met by phone on 16 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

documented TOP (again), and explained how parsing is initiated and how it actually works
series operator (...) now picks a monotonic function when using single characters as endpoints
STD can now catch duplicates involving protos as well as onlys
STD no longer advises removal of parens on spaceless sub() declaration
mostly advised sorear and pmichaud
Stefan is finishing the boostrap of the STD parser
also working on adding a parallel NFA and DFA engine
no, he doesn't want to generate all the states in advance
it works faster lazily

Allison:

working on chroot environments with something more secure than chroot
relevant to building Parrot packages
looking at some bugs for Will

Patrick:

Rakudo developers decided not to make extra special effort to make a June release of Rakudo Star
the calendar works against us
the new date for the release is July 29
we're I comfortable with hitting that target
we won't be happy with the results of moving heaven and earth to release in June
there are lots of advantages
one disadvantage is not having Rakudo Star at YAPC::NA
one big advantage is using the supported Parrot 2.6 release as the basis
I'll write a post outlining the plan in the next couple of days
otherwise working on lists and interators in Perl 6 and Rakudo
after deciding to make iterators immutable, Larry and I realized that solves many problems
everything works out as plain as day after that
very happy with that design
the incorrect assumptions of the old model were pervasive
replacing the old pieces is taking a while, which is no surprise
this approach feels right though
the new branch does things no previous version could do
slices work much better, for example
metaoperators work properly
map is lazy
slurpy arguments in lists are lazy by default
no weird binding or action at a distance problems
plenty of changes to Associative and Positional roles
those are now super clean and may be lazy
more features work
~30 failing tests (not test files, just tests) now, ~500 last night
most of the current failures are minor
will try to merge the branch before the release
replacing lots of ugly code with fewer lines of elegant code
Jonathan and others have worked on lots of other pieces
adding plenty of new features
looking forward to tomorrow's release

editing the Rakudo book
moving the Rakudo release date may let us have a printed book available about the same time
depends on how much there is left to write

Perl 6 Design Minutes for 09 June 2010

Thu, 24 Jun 2010 13:16:02 GMT

The Perl 6 design team met by phone on 09 June 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

not much spec change this week
figured out a syntax for a regex block to return more than one cursor
based on gather/take
in STD hacking, continued to assist Stefan O'Rear in getting STD bootstrapped via viv
now that it's bootstrapped, we're refactoring things that make sense now
we're now starting to move bits of Cursor code from Perl 5 into Perl 6
refactoring the grammar for sanity of design
started upgrading STD to normal Perl 6 syntax where it previously catered to gimme5's limitations
for example, switched STD's old .<_from> and .<_pos> hash lookups to using .from and .pos accessors
started the prep work for moving EXPR out of STD to make it generally available to any grammar wanting operator precedence
in STD parsing, made Perl 5 $< detection have a longer token to avoid confusion with match variables
STD no longer attempts two-terms detection on infix_circumfix_meta_operator
STD now parses >>R~<< correctly, or at least dwimmily
STD doesn't complain about P5isms in printf formats like "%{$count}s"
STD was parsing /m and /s with the opposite semantics
termish now localizes $*MULTINESS in its scope so that inner declarations aren't accidentally multified
STD now carps about package Foo; as a Perl 5 construct

Allison:

talked to Chris Shiflett, a PHP developer, on someone from the PHP community to sit on the Parrot board
will be in the US for a few weeks

Patrick:

working on list simplification
had a couple of breakthrough ideas on Monday
working on the implementation now
worked out inversion lists for character class matching in regexes
will make them faster, especially with long ranges of character classes
fixed a half-dozen tickets in RT
fixed Rakudo hash constructors
fixed an intermittent bug with colon-pair signatures
two possible parses exist in STD, but we removed an unneeded one in Rakudo
fixed a bug with Parrot's exit opcode
NQP and PAST needed an update not to cheat with PASM constants
I fixed that too
Vasily added multisub and multimethod support to NQP, that was a big plus
fixed the ** quantifier in regexes to understand surrounding whitespace
regex engine tried to match beyond the end of a string, so I added guards for that
will work on lists furiously before the next release
I don't think it'll take long
closures are next, hope to have those in place by the weekend

released a new version of Pod::PseudoPod::LaTeX to support the various books in progress

Perl 6 Design Minutes for 02 June 2010

Tue, 22 Jun 2010 02:13:44 GMT

The Perl 6 design team met by phone on 02 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

mostly, I supported sorear in bootstrapping STD to use viv instead of gimme5
his stage 2 and stage 3 now output identical Perl 5 versions of STD
produces a huge amount of warnings
appears to require Perl 5.12 at the moment
working on both of those
S03 refines hyper dwimminess to be more like APL, with modular semantics
S02 refines Blobs to simply be immutable Bufs, with similar generic characteristics
S02 now describes native blob types
implemented post-declaration checks for BEGIN and use, since those can't wait for end of file
STD no longer loses existing bindings when we go to a sublanguage
STD now uses $*GOAL variable only as informative, never as a "stopper"
instead, we create a <stopper> rule for $*GOAL if necessary
can check for that only, instead of that or $*GOAL
answering lots of questions on how STD and viv work besides that

Allison:

did a lot of research on graph color algorithms for register usage algorithms
will finish my finals on Monday

Will:

trying to herd the discussion of dynop libraries
a recent branch to close an old ticket broke a lot of assumptions
some bugs have become more visible because of these changes
hope to get that cleaned up this week

Allison:

I liked your suggestion of bringing back the getstderr and related opcodes

Will:

trying to resurrect Partcl
stuck on a TT #389 closing issue
not sure how to fix that, the way things are now

Patrick:

working on the iterator and list design
brainstorming the implementation
will implement somethine one way or another this week
people keep implementing workarounds for the current system
they'll bite us eventually
Moritz and I worked on making the regex engine returning real Perl 6 objects
that mostly works
exposes some places where lists don't work exactly right
the workarounds there made me replan the list and iterator implementation
answered some questions online
Jonathan added a better backtrace algorithm for Rakudo
reports Perl 6 source lines instead of PIR lines
I'll review his code
think I can borrow it for NQP for all HLLs
Jonathan reports that it was a lot easier in NQP than PIR

trying to answer a few Parrot design questions
looking at the continuation of design from Perl 1 - 4 to Perl 5 and Perl 6
hope to have coding time soon

Perl 6 Design Minutes for 26 May 2010

Sun, 20 Jun 2010 20:11:17 GMT

The Perl 6 design team met by phone on 26 May 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

:() syntax is now always signature
we now use foofix:[...] as the general op form instead of foofix:(...)
refactored the sematics of :nth and :x
:nth() now only ever takes a monotonically increasing list
S03 now explains how "not-raising" works on != and ne
it now basically matches the intuitions of an English speaker via HOP definition of negate metaop
STD sometimes didn't require semi between statements
statement modifiers are expression terminators but not valid statement terminators
an unexpected statement modifier word like if could terminate one statement and start another
fixed up backslashes in character classes to allow \s etc and reject \u etc
STD was accidentally using the same lexpad for different multis
Cursor now treats :() on name extension as a signature always, never as a categorical
we shouldn't introduce the stopper for circumfix until we're in the circumfix, or we can't use the same char on both ends
placeholder messages error messages are now much more informative and correct
we now disallow use of placeholder after same variable has been used as a non-placeholder, even for an outer reference
renamed add_macro (which it doesn't) to add_categorical (which it does)
participating frequently in discussions on semantics both on irc and p6l
working closely with sorear++ as he brings viv closer to bootstrapping, yay!
soon can bootstrap past gimme5

Allison:

worked on Pynie this week in my limited spare time
one goal is to generate the parser directly from the Python grammar
wrote a small, lightweight PEG parser which generates a match tree from the Python 3 grammar
can generate a lexer directly
right now it creates a parse tree
looks similar to the match nodes of NQP-rx
dumps out a tree to the PIR parser
working on PaFo elections for next year, but trying to delegate those
will have more time after June 7

Will:

working on Perl 6 advent tests
many more people are doing more work than me
liasing with Rakudo folks for any important Parrot bugs before the Rakudo Star release
my current direction there is "don't break anything"

Patrick:

sorear added hash flattening to NQP
lots of work on closures in PAST and NQP
they properly clone
fixes some lexical problems
need to get that to work in Rakudo
that's tougher; Rakudo has to wrap Parrot subs
wrapper object needs cloning as well, along with its attributes
we'll add a new PAST node type to help
that node understands contexts
essentially a way to add void context optimizations to your AST
that solves many problems in Rakudo beyond closures
added a setting into NQP along with its test suite
not automatically loaded, but available
contains standard hash and array methods
Parrot's ops2c project uses those
other people can update and enhance that setting as necessary
NQP also has the ability to parse type names
NQP doesn't do anything with them yet
eventually they'll allow the use of multis
cleaning up some NQP bugs regarding lexicals and package storage of subs
Bruce Keeler enabled variable interpolations in regexes
working on some refactorings to simplify that approach
works in NQP and Rakudo now
that's a feature we've never had before
Rakudo's REPL now works better, thanks to sorear
HLLCompiler now written more in NQP as part of that
NQP now can do eval
NQP remembers lexicals in interactive mode now
adding that to Rakudo is more complex
working on that
pleased with the progress on #perl6

reviewing long term plans for GC and Lorito
should have more time free soon

Perl 6 Design Minutes for 19 May 2010

Wed, 16 Jun 2010 22:10:54 GMT

The Perl 6 design team met by phone on 19 May 2010. Larry, Will, and chromatic attended. Patrick added his notes later.

Larry:

S03 makes more explicit that doctrine that ~~ topicalizes, and removes smartmatch table fossils that automatically fall out from that
S05 renames 'accent' to 'mark' for better Unicode conformance
:a and :aa changed to :m and :mm
S05 disrequires retroactive semantics on :samecase and :samemark
the method form must now explicitly add case or mark modifiers to the pattern
regularized mm// to ms// to avoid confusion with new :m ignoremark option
STD now does a bit better at diagnosing bogus ??!! constructs of various sorts
STD now correctly adds operators to symbol tables as subs
CORE.setting now has protos of all the operators so they can be recognized as subs too
Cursor now canonicalize operator names in the symbol table
btw, not quite like specced
STD now reads user's mind on 'Str $toto' to intuit missing declarator
STD now properly diagnoses a typename between routine declarator and sub name

Will:

working on code for Carl Masak, trying to get his poker code example running on Rakudo
both fun and frustrating
some stuff doesn't quite work yet
going through the Advent examples
adding them to spectests
make sure we won't regress on such public examples
other people are helping with that now

will get back to editing the Rakudo book soon
hope to have it in print by YAPC, but no guarantee

Patrick:

fixed closures in NQP, as a precursor for fixing them in Rakudo
worked with sorear on REPL in Rakudo and PCT in general
ported the NQP "standard library" done by japhb++, bacek++, and many others into the nqp-rx repository and made it part of the standard build sequence for nqp and Parrot
decided we need a new "context sensitive" node type in PAST, will be used to create proper closures and to handle sink context
worked with bacek on adding better multimethod support to PAST and nqp-rx
discovered a problem with lexical subs in NQP being automatically entered into the package namespace (and some existing code relying on this behavior)
did some initial fixes to at least get things entered properly, but a complete fix may require a deprecation cycle
plan to review others' patches this week
plan to fix REPL, closures, and sink context in Rakudo (since those are currently large pain points)
plan to work on loops and iterators after that

Perl 6 Design Minutes for 12 May 2010

Wed, 16 Jun 2010 02:06:23 GMT

The Perl 6 design team met by phone on 12 May 2010. Larry, Allison, Patrick, and Will attended.

Larry:

clarified usage of brackets around infixes
added various 128-bit types to the spec; we might make them arbitrarily extensible via role
at least LLVM could support this, even to non-powers-of-two sizes
modernized the paleolithic grammatical category description in S02
STD now uses double-quote rules for interpolating @foo[] into regex
STD now gives better message on 1__3
added the specced 128-bit types to CORE.setting
added minmax function to CORE.setting
implemented circumfix:«X Y» as grammar derivation
currently only allows a >> inside
now also recognizes foofix:("\x[face]") and foofix:("\c[YOUR CHARACTER HERE]") without actually evaluating
playing with factoring yaml out of gimme5, since viv is not likely to go that route.
mostly just answered a lot of questions on irc
egged people on about concurrency issues

Patrick:

thought on handling closures properly
have a solution, just need some time to implement
discussion on changes to CodeString
work on compiler toolkit to avoid CodeString, using StringBuilder instead where possible, in PCT, NQP, and rakudo. Pretty easy, no downstream projects block on a deprecation issue
after that, lists
also been answering questions on interactive mode (REPL) for rakudo et al. (the issue with losing lexicals)

Allison:

resolved the git conversation pretty well (for Parrot's repo migration)
worked on a pure PEG parser (following the paper), straight PIR, single day; now self-parsing. Interesting project, is lightweight. currently has memoization, but that might not be right for us because of backtracking. With some more effort, could probably handle EBNF form (useful for python)
could be setup for developer status for Debian which will improve our packaging status for Debian and Ubuntu

Will:

Parrot CodeString performance improvements
we're definitely faster in branch, but some feedback from pmichaud should help us clean up the API a bit as well, look for those to hit trunk in the next few days
Parrot makefile deps cleanup

Perl 6 Design Minutes for 05 May 2010

Mon, 10 May 2010 06:11:34 GMT

The Perl 6 design team met by phone on 05 May 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

various spec updates, some major
removed p5=> description because it's not supported in core
deleted self:sort construct because self isn't a real syntactic category
explained Perl patterns in terms of PEGs, and spec'ed tiebreaking rules explicitly
last but not least, finally purveyed the long-threatened revamp of proto to keep routine and method semantics similar
they all now work much more like the multiple dispatch semantics currently used by STD, where we always call the proto first
the proto is then always in charge of the actual multiple dispatch; it can of course delegate that
and the default for a null body corresponds closely to current semantics
in hacking news, the lexer generator mislaid any alternative that was a bare . pattern, so cursor_fate never called its alternative, oops
took me a long time to run that one down, because it resulted in a horrendous backtrack causing mysterious misplaced errors
revamped character class parsing to be more helpful and correct
STD now check a normal regex bracket's innards for old-school character class, and warns if found
added a .looks_like_cclass method to Cursor to detect most accidental uses of P5 ranges
some valid P6 brackets will complain, but the workarounds are easy
just put whitespace on both ends is one way
removed a few of these old-school-ish character classes from STD
changed :tr language to :cc language since character classes share it
(translation pays more attention to ordering, but the language is the same)
turned out parsing character classes discovered issues in STD; various character classes needed to backslash # that would otherwise be a comment
to that end, we now allow \# in character classes instead of misparsing as unspace
if we find an invalid - in a regex, we now presume we're in an old-school character class and fail with a sorry instead of a panic to give the character class code a shot at it
STD now uses ~ syntax for regex brackets to set $*GOAL correctly
cleaned up recursive panic detection; it was possible to get both false positives and negatives before
STD shouldn't use 'note' to emit a panic inside a suppose because that leaks the message that should be trapped
STD now suppresses duplicate sorry messages more correctly
sorry no longer uses panic in a supposition, but dies directly to throw the exception to the suppose's try block
STD now allows subscripts on regex variables so $x[0] isn't taken as a character class; still needs speccing

Patrick:

can we make them consistent?

Larry:

historically S05 has allowed bare arrays to mean interpolation

Patrick:

we've never had a working implementation of that

Larry:

a bare @ would be illegal

Patrick:

it's currently illegal

Larry:

you'd have to backslash it to match part of an email address
it's not like the @ alternations are a big deal one way or another
that'd be a little more consistent
I forced it to think of the sigil as $ than what it really is

Patrick:

after seeing how Jonathan et all did interpolation for quoted strings, I thought we should do the same thing in regexes

Larry:

STD now has a partial fix to prevent leakage of ::T from role signatures
unfortunately, the current fix will lose signatures of file-scoped generic roles
this probably has to do with not knowing whether we're really going to want a new pad; unfortunately we'd have to look ahead to know that currently
various other minor tweaks and bug fixes in STD and Cursor

Patrick:

mostly responding to messages and reports
should be able to get back to coding full-time and online for the next week
plan to resolve the list and closure issues with NQP and Rakudo
will answer other questions and try to keep other people productive
planning for the Rakudo Star release on June

Allison:

busy with the last week of classes
spent most of it writing a little language with PCT
it was easy to use and easy to swap the stages of PCT
I remembered what Patrick did with LOLCODE
also had a discussion of source code control systems
next week should be more productive
need to work more closely with Debian packagers to get packages into Debian

Will:

cleaning out as many deprecations in Parrot as possible
trying to improve the speed of CodeString after the immutable STRINGs merge
bundling lots of little concats helps
hope to merge in an optimization branch for that by the weekend
want to make that faster or less memory intensive
may require the use of a new StringBuilder for Parrot
hopefully will result in a faster Rakudo build

Patrick:

I've never seen CodeString take a long time
unless you run into memory problems
* discussion of the StringBuilder PMC *

still working on optimizations, particularly CodeString
looking at more PBC and PBC-building optimizations
PBC size went down dramatically and startup improved for Rakudo
should have that much faster for the 2.4 release
will poke at GC tasks starting next week

Perl 6 Design Minutes for 28 April 2010

Sun, 2 May 2010 01:07:12 GMT

The Perl 6 design team met by phone on 28 April 2010. Larry, Allison, Jerry, Will, and chromatic attended.

Larry:

caught up on a week's worth of backlog
made a few spec tweaks
discussed them with other people
trying to make error messages more awesome in STD
working on the ability to parse the insides of character classes
STD doesn't like parsing itself recursively there
need to iron out a few things
enum names can now be variables

Allison:

Debian packages ready to ship to Debian sponsors
putting together a list of GC tasks
cleaned out the existing page, have the big things listed
trying to decide which tasks to do first
doing a lot of reading and research
my little language project is due on Monday
HLLCompiler was enormously useful
will start working on the GC stuff next week
should also start a fresh pass through the ticket queue
added a workaround for the final remaining TT #389 bug
Jonathan had a test case

Will:

tried to focus on getting Rakudo blockers removed

spent some time getting Rakudo to work with trunk
will need a Rakudo guts hacker for the last part
worked on the compact_string revamp branch with Vasily
merged now
that makes trunk about 12% faster than the 2.3.0 release
will work on a few Rakudo profiles once it works with trunk again
expect at least a 5% performance improvement there
have some other ideas, but won't do them without profiling first
came up with a scheme to reduce PBC size by coalescing strings
Peter Lobsinger is exploring that