Recent blog entries for chromatic

Perl 6 Design Minutes for 30 June 2010

The Perl 6 design team met by phone on 30 June 2010. Allison, Patrick, and chromatic attended.

Allison:

  • working on Parrot packages for Debian experimental
  • seems like a good idea to do that before the 2.6 supported release
  • there was also a request for Rakudo packages
  • not sure if I'm the best person to do it

Patrick:

  • I'm sure we should package Rakudo Star

Allison:

  • Debian had a packager for those, but I haven't looked at the packages
  • this'd be an early run of what we'll do with Rakudo Star

Patrick:

  • we're not quite ready for packaging that yet
  • maybe a couple of weeks
  • finished the List and Iterator types for the #30 release
  • adjusted Rakudo's Associative and Positional roles
  • much cleaner implementation now
  • that'll require a few small spec changes
  • redid Rakudo's container types
  • more robust
  • preparing for autovivification of hashes and arrays
  • expect to finish those in the next couple of days
  • there was no container model previously; the code was consequently crufty
  • lots of cleanup of incorrect assumptions
  • Rakudo lists are now properly lazy
  • comment syntax fixed
  • ROADMAP updated
  • fixed the meaning of Nil; it's defined, not undefined
  • added the sink prefix (?)
  • fixed setting of $!
  • started fixing bugs and closing tickets on Monday, did 15 or 20
  • mostly already fixed in the previous couple of weeks
  • looking at the implementation of the series operator
  • spec is self-contradictory or ambiguous or both
  • waiting for Larry's clarification
  • fixed a bug in $*ARGFILES
  • had a nice contribution of that implementation last week
  • that behavior works on any set of files, not just those on the command line
  • working on autoviv
  • have some regex backtracking bugs to fix
  • will work on closures after that
  • put together three new YAPC presentations
  • the Rakudo Star presentation will become a video cast or a blog post or both

c:

  • worked on a slew of Parrot optimizations for Rakudo
  • have a few more to go
  • might have to create a Rakudo branch temporarily
  • will try to help merge the new GC
  • working on a metamodel for Parrot objects, informed by Perl 6 and Moose

Syndicated 2010-07-03 08:13:30 from pudge

Modern Perl: The (Draft) Book

This took longer than I expected, but the draft of the Modern Perl book is available for review. I'm especially interested in hearing from people who don't consider themselves expert Perl 5 programmers. The goal of the book is to explain how Perl 5 works (and how to write Perl 5 effectively) to help novices become adepts.

Syndicated 2010-06-28 23:43:33 from pudge

Perl 6 Design Minutes for 16 June 2010

The Perl 6 design team met by phone on 16 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

  • documented TOP (again), and explained how parsing is initiated and how it actually works
  • series operator (...) now picks a monotonic function when using single characters as endpoints
  • STD can now catch duplicates involving protos as well as onlys
  • STD no longer advises removal of parens on spaceless sub() declaration
  • mostly advised sorear and pmichaud
  • Stefan is finishing the boostrap of the STD parser
  • also working on adding a parallel NFA and DFA engine
  • no, he doesn't want to generate all the states in advance
  • it works faster lazily

Allison:

  • working on chroot environments with something more secure than chroot
  • relevant to building Parrot packages
  • looking at some bugs for Will

Patrick:

  • Rakudo developers decided not to make extra special effort to make a June release of Rakudo Star
  • the calendar works against us
  • the new date for the release is July 29
  • we're I comfortable with hitting that target
  • we won't be happy with the results of moving heaven and earth to release in June
  • there are lots of advantages
  • one disadvantage is not having Rakudo Star at YAPC::NA
  • one big advantage is using the supported Parrot 2.6 release as the basis
  • I'll write a post outlining the plan in the next couple of days
  • otherwise working on lists and interators in Perl 6 and Rakudo
  • after deciding to make iterators immutable, Larry and I realized that solves many problems
  • everything works out as plain as day after that
  • very happy with that design
  • the incorrect assumptions of the old model were pervasive
  • replacing the old pieces is taking a while, which is no surprise
  • this approach feels right though
  • the new branch does things no previous version could do
  • slices work much better, for example
  • metaoperators work properly
  • map is lazy
  • slurpy arguments in lists are lazy by default
  • no weird binding or action at a distance problems
  • plenty of changes to Associative and Positional roles
  • those are now super clean and may be lazy
  • more features work
  • ~30 failing tests (not test files, just tests) now, ~500 last night
  • most of the current failures are minor
  • will try to merge the branch before the release
  • replacing lots of ugly code with fewer lines of elegant code
  • Jonathan and others have worked on lots of other pieces
  • adding plenty of new features
  • looking forward to tomorrow's release

c:

  • editing the Rakudo book
  • moving the Rakudo release date may let us have a printed book available about the same time
  • depends on how much there is left to write

Syndicated 2010-06-26 17:07:30 from pudge

Perl 6 Design Minutes for 09 June 2010

The Perl 6 design team met by phone on 09 June 2010. Larry, Allison, Patrick, and chromatic attended.

Larry:

  • not much spec change this week
  • figured out a syntax for a regex block to return more than one cursor
  • based on gather/take
  • in STD hacking, continued to assist Stefan O'Rear in getting STD bootstrapped via viv
  • now that it's bootstrapped, we're refactoring things that make sense now
  • we're now starting to move bits of Cursor code from Perl 5 into Perl 6
  • refactoring the grammar for sanity of design
  • started upgrading STD to normal Perl 6 syntax where it previously catered to gimme5's limitations
  • for example, switched STD's old.<_from> and.<_pos> hash lookups to using.from and.pos accessors
  • started the prep work for moving EXPR out of STD to make it generally available to any grammar wanting operator precedence
  • in STD parsing, made Perl 5 $< detection have a longer token to avoid confusion with match variables
  • STD no longer attempts two-terms detection on infix_circumfix_meta_operator
  • STD now parses >>R~<< correctly, or at least dwimmily
  • STD doesn't complain about P5isms in printf formats like "%{$count}s"
  • STD was parsing/m and/s with the opposite semantics
  • termish now localizes $*MULTINESS in its scope so that inner declarations aren't accidentally multified
  • STD now carps about package Foo; as a Perl 5 construct

Allison:

  • talked to Chris Shiflett, a PHP developer, on someone from the PHP community to sit on the Parrot board
  • will be in the US for a few weeks

Patrick:

  • working on list simplification
  • had a couple of breakthrough ideas on Monday
  • working on the implementation now
  • worked out inversion lists for character class matching in regexes
  • will make them faster, especially with long ranges of character classes
  • fixed a half-dozen tickets in RT
  • fixed Rakudo hash constructors
  • fixed an intermittent bug with colon-pair signatures
  • two possible parses exist in STD, but we removed an unneeded one in Rakudo
  • fixed a bug with Parrot's exit opcode
  • NQP and PAST needed an update not to cheat with PASM constants
  • I fixed that too
  • Vasily added multisub and multimethod support to NQP, that was a big plus
  • fixed the ** quantifier in regexes to understand surrounding whitespace
  • regex engine tried to match beyond the end of a string, so I added guards for that
  • will work on lists furiously before the next release
  • I don't think it'll take long
  • closures are next, hope to have those in place by the weekend

c:

  • released a new version of Pod::PseudoPod::LaTeX to support the various books in progress

Syndicated 2010-06-24 12:24:33 from pudge

Perl 6 Design Minutes for 02 June 2010

The Perl 6 design team met by phone on 02 June 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

  • mostly, I supported sorear in bootstrapping STD to use viv instead of gimme5
  • his stage 2 and stage 3 now output identical Perl 5 versions of STD
  • produces a huge amount of warnings
  • appears to require Perl 5.12 at the moment
  • working on both of those
  • S03 refines hyper dwimminess to be more like APL, with modular semantics
  • S02 refines Blobs to simply be immutable Bufs, with similar generic characteristics
  • S02 now describes native blob types
  • implemented post-declaration checks for BEGIN and use, since those can't wait for end of file
  • STD no longer loses existing bindings when we go to a sublanguage
  • STD now uses $*GOAL variable only as informative, never as a "stopper"
  • instead, we create a <stopper> rule for $*GOAL if necessary
  • can check for that only, instead of that or $*GOAL
  • answering lots of questions on how STD and viv work besides that

Allison:

  • did a lot of research on graph color algorithms for register usage algorithms
  • will finish my finals on Monday

Will:

  • trying to herd the discussion of dynop libraries
  • a recent branch to close an old ticket broke a lot of assumptions
  • some bugs have become more visible because of these changes
  • hope to get that cleaned up this week

Allison:

  • I liked your suggestion of bringing back the getstderr and related opcodes

Will:

  • trying to resurrect Partcl
  • stuck on a TT #389 closing issue
  • not sure how to fix that, the way things are now

Patrick:

  • working on the iterator and list design
  • brainstorming the implementation
  • will implement somethine one way or another this week
  • people keep implementing workarounds for the current system
  • they'll bite us eventually
  • Moritz and I worked on making the regex engine returning real Perl 6 objects
  • that mostly works
  • exposes some places where lists don't work exactly right
  • the workarounds there made me replan the list and iterator implementation
  • answered some questions online
  • Jonathan added a better backtrace algorithm for Rakudo
  • reports Perl 6 source lines instead of PIR lines
  • I'll review his code
  • think I can borrow it for NQP for all HLLs
  • Jonathan reports that it was a lot easier in NQP than PIR

c:

  • trying to answer a few Parrot design questions
  • looking at the continuation of design from Perl 1 - 4 to Perl 5 and Perl 6
  • hope to have coding time soon

Syndicated 2010-06-22 01:12:29 from pudge

Perl 6 Design Minutes for 26 May 2010

The Perl 6 design team met by phone on 26 May 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

  • :() syntax is now always signature
  • we now use foofix:[...] as the general op form instead of foofix:(...)
  • refactored the sematics of:nth and:x
  • :nth() now only ever takes a monotonically increasing list
  • S03 now explains how "not-raising" works on != and ne
  • it now basically matches the intuitions of an English speaker via HOP definition of negate metaop
  • STD sometimes didn't require semi between statements
  • statement modifiers are expression terminators but not valid statement terminators
  • an unexpected statement modifier word like if could terminate one statement and start another
  • fixed up backslashes in character classes to allow \s etc and reject \u etc
  • STD was accidentally using the same lexpad for different multis
  • Cursor now treats:() on name extension as a signature always, never as a categorical
  • we shouldn't introduce the stopper for circumfix until we're in the circumfix, or we can't use the same char on both ends
  • placeholder messages error messages are now much more informative and correct
  • we now disallow use of placeholder after same variable has been used as a non-placeholder, even for an outer reference
  • renamed add_macro (which it doesn't) to add_categorical (which it does)
  • participating frequently in discussions on semantics both on irc and p6l
  • working closely with sorear++ as he brings viv closer to bootstrapping, yay!
  • soon can bootstrap past gimme5

Allison:

  • worked on Pynie this week in my limited spare time
  • one goal is to generate the parser directly from the Python grammar
  • wrote a small, lightweight PEG parser which generates a match tree from the Python 3 grammar
  • can generate a lexer directly
  • right now it creates a parse tree
  • looks similar to the match nodes of NQP-rx
  • dumps out a tree to the PIR parser
  • working on PaFo elections for next year, but trying to delegate those
  • will have more time after June 7

Will:

  • working on Perl 6 advent tests
  • many more people are doing more work than me
  • liasing with Rakudo folks for any important Parrot bugs before the Rakudo Star release
  • my current direction there is "don't break anything"

Patrick:

  • sorear added hash flattening to NQP
  • lots of work on closures in PAST and NQP
  • they properly clone
  • fixes some lexical problems
  • need to get that to work in Rakudo
  • that's tougher; Rakudo has to wrap Parrot subs
  • wrapper object needs cloning as well, along with its attributes
  • we'll add a new PAST node type to help
  • that node understands contexts
  • essentially a way to add void context optimizations to your AST
  • that solves many problems in Rakudo beyond closures
  • added a setting into NQP along with its test suite
  • not automatically loaded, but available
  • contains standard hash and array methods
  • Parrot's ops2c project uses those
  • other people can update and enhance that setting as necessary
  • NQP also has the ability to parse type names
  • NQP doesn't do anything with them yet
  • eventually they'll allow the use of multis
  • cleaning up some NQP bugs regarding lexicals and package storage of subs
  • Bruce Keeler enabled variable interpolations in regexes
  • working on some refactorings to simplify that approach
  • works in NQP and Rakudo now
  • that's a feature we've never had before
  • Rakudo's REPL now works better, thanks to sorear
  • HLLCompiler now written more in NQP as part of that
  • NQP now can do eval
  • NQP remembers lexicals in interactive mode now
  • adding that to Rakudo is more complex
  • working on that
  • pleased with the progress on #perl6

c:

  • reviewing long term plans for GC and Lorito
  • should have more time free soon

Syndicated 2010-06-20 19:40:02 from pudge

Perl 6 Design Minutes for 19 May 2010

The Perl 6 design team met by phone on 19 May 2010. Larry, Will, and chromatic attended. Patrick added his notes later.

Larry:

  • S03 makes more explicit that doctrine that ~~ topicalizes, and removes smartmatch table fossils that automatically fall out from that
  • S05 renames 'accent' to 'mark' for better Unicode conformance
  • :a and:aa changed to:m and:mm
  • S05 disrequires retroactive semantics on:samecase and:samemark
  • the method form must now explicitly add case or mark modifiers to the pattern
  • regularized mm// to ms// to avoid confusion with new:m ignoremark option
  • STD now does a bit better at diagnosing bogus ??!! constructs of various sorts
  • STD now correctly adds operators to symbol tables as subs
  • CORE.setting now has protos of all the operators so they can be recognized as subs too
  • Cursor now canonicalize operator names in the symbol table
  • btw, not quite like specced
  • STD now reads user's mind on 'Str $toto' to intuit missing declarator
  • STD now properly diagnoses a typename between routine declarator and sub name

Will:

  • working on code for Carl Masak, trying to get his poker code example running on Rakudo
  • both fun and frustrating
  • some stuff doesn't quite work yet
  • going through the Advent examples
  • adding them to spectests
  • make sure we won't regress on such public examples
  • other people are helping with that now

c:

  • will get back to editing the Rakudo book soon
  • hope to have it in print by YAPC, but no guarantee

Patrick:

  • fixed closures in NQP, as a precursor for fixing them in Rakudo
  • worked with sorear on REPL in Rakudo and PCT in general
  • ported the NQP "standard library" done by japhb++, bacek++, and many others into the nqp-rx repository and made it part of the standard build sequence for nqp and Parrot
  • decided we need a new "context sensitive" node type in PAST, will be used to create proper closures and to handle sink context
  • worked with bacek on adding better multimethod support to PAST and nqp-rx
  • discovered a problem with lexical subs in NQP being automatically entered into the package namespace (and some existing code relying on this behavior)
  • did some initial fixes to at least get things entered properly, but a complete fix may require a deprecation cycle
  • plan to review others' patches this week
  • plan to fix REPL, closures, and sink context in Rakudo (since those are currently large pain points)
  • plan to work on loops and iterators after that

Syndicated 2010-06-16 21:38:09 from pudge

Perl 6 Design Minutes for 12 May 2010

The Perl 6 design team met by phone on 12 May 2010. Larry, Allison, Patrick, and Will attended.

Larry:

  • clarified usage of brackets around infixes
  • added various 128-bit types to the spec; we might make them arbitrarily extensible via role
  • at least LLVM could support this, even to non-powers-of-two sizes
  • modernized the paleolithic grammatical category description in S02
  • STD now uses double-quote rules for interpolating @foo[] into regex
  • STD now gives better message on 1__3
  • added the specced 128-bit types to CORE.setting
  • added minmax function to CORE.setting
  • implemented circumfix:«X Y» as grammar derivation
  • currently only allows a >> inside
  • now also recognizes foofix:("\x[face]") and foofix:("\c[YOUR CHARACTER HERE]") without actually evaluating
  • playing with factoring yaml out of gimme5, since viv is not likely to go that route.
  • mostly just answered a lot of questions on irc
  • egged people on about concurrency issues

Patrick:

  • thought on handling closures properly
  • have a solution, just need some time to implement
  • discussion on changes to CodeString
  • work on compiler toolkit to avoid CodeString, using StringBuilder instead where possible, in PCT, NQP, and rakudo. Pretty easy, no downstream projects block on a deprecation issue
  • after that, lists
  • also been answering questions on interactive mode (REPL) for rakudo et al. (the issue with losing lexicals)

Allison:

  • resolved the git conversation pretty well (for Parrot's repo migration)
  • worked on a pure PEG parser (following the paper), straight PIR, single day; now self-parsing. Interesting project, is lightweight. currently has memoization, but that might not be right for us because of backtracking. With some more effort, could probably handle EBNF form (useful for python)
  • could be setup for developer status for Debian which will improve our packaging status for Debian and Ubuntu

Will:

  • Parrot CodeString performance improvements
  • we're definitely faster in branch, but some feedback from pmichaud should help us clean up the API a bit as well, look for those to hit trunk in the next few days
  • Parrot makefile deps cleanup

Syndicated 2010-06-16 01:47:02 from pudge

Perl 6 Design Minutes for 05 May 2010

The Perl 6 design team met by phone on 05 May 2010. Larry, Allison, Patrick, Will, and chromatic attended.

Larry:

  • various spec updates, some major
  • removed p5=> description because it's not supported in core
  • deleted self:sort construct because self isn't a real syntactic category
  • explained Perl patterns in terms of PEGs, and spec'ed tiebreaking rules explicitly
  • last but not least, finally purveyed the long-threatened revamp of proto to keep routine and method semantics similar
  • they all now work much more like the multiple dispatch semantics currently used by STD, where we always call the proto first
  • the proto is then always in charge of the actual multiple dispatch; it can of course delegate that
  • and the default for a null body corresponds closely to current semantics
  • in hacking news, the lexer generator mislaid any alternative that was a bare. pattern, so cursor_fate never called its alternative, oops
  • took me a long time to run that one down, because it resulted in a horrendous backtrack causing mysterious misplaced errors
  • revamped character class parsing to be more helpful and correct
  • STD now check a normal regex bracket's innards for old-school character class, and warns if found
  • added a.looks_like_cclass method to Cursor to detect most accidental uses of P5 ranges
  • some valid P6 brackets will complain, but the workarounds are easy
  • just put whitespace on both ends is one way
  • removed a few of these old-school-ish character classes from STD
  • changed:tr language to:cc language since character classes share it
  • (translation pays more attention to ordering, but the language is the same)
  • turned out parsing character classes discovered issues in STD; various character classes needed to backslash # that would otherwise be a comment
  • to that end, we now allow \# in character classes instead of misparsing as unspace
  • if we find an invalid - in a regex, we now presume we're in an old-school character class and fail with a sorry instead of a panic to give the character class code a shot at it
  • STD now uses ~ syntax for regex brackets to set $*GOAL correctly
  • cleaned up recursive panic detection; it was possible to get both false positives and negatives before
  • STD shouldn't use 'note' to emit a panic inside a suppose because that leaks the message that should be trapped
  • STD now suppresses duplicate sorry messages more correctly
  • sorry no longer uses panic in a supposition, but dies directly to throw the exception to the suppose's try block
  • STD now allows subscripts on regex variables so $x[0] isn't taken as a character class; still needs speccing

Patrick:

  • can we make them consistent?

Larry:

  • historically S05 has allowed bare arrays to mean interpolation

Patrick:

  • we've never had a working implementation of that

Larry:

  • a bare @ would be illegal

Patrick:

  • it's currently illegal

Larry:

  • you'd have to backslash it to match part of an email address
  • it's not like the @ alternations are a big deal one way or another
  • that'd be a little more consistent
  • I forced it to think of the sigil as $ than what it really is

Patrick:

  • after seeing how Jonathan et all did interpolation for quoted strings, I thought we should do the same thing in regexes

Larry:

  • STD now has a partial fix to prevent leakage of::T from role signatures
  • unfortunately, the current fix will lose signatures of file-scoped generic roles
  • this probably has to do with not knowing whether we're really going to want a new pad; unfortunately we'd have to look ahead to know that currently
  • various other minor tweaks and bug fixes in STD and Cursor

Patrick:

  • mostly responding to messages and reports
  • should be able to get back to coding full-time and online for the next week
  • plan to resolve the list and closure issues with NQP and Rakudo
  • will answer other questions and try to keep other people productive
  • planning for the Rakudo Star release on June

Allison:

  • busy with the last week of classes
  • spent most of it writing a little language with PCT
  • it was easy to use and easy to swap the stages of PCT
  • I remembered what Patrick did with LOLCODE
  • also had a discussion of source code control systems
  • next week should be more productive
  • need to work more closely with Debian packagers to get packages into Debian

Will:

  • cleaning out as many deprecations in Parrot as possible
  • trying to improve the speed of CodeString after the immutable STRINGs merge
  • bundling lots of little concats helps
  • hope to merge in an optimization branch for that by the weekend
  • want to make that faster or less memory intensive
  • may require the use of a new StringBuilder for Parrot
  • hopefully will result in a faster Rakudo build

Patrick:

  • I've never seen CodeString take a long time
  • unless you run into memory problems
  • * discussion of the StringBuilder PMC *

c:

  • still working on optimizations, particularly CodeString
  • looking at more PBC and PBC-building optimizations
  • PBC size went down dramatically and startup improved for Rakudo
  • should have that much faster for the 2.4 release
  • will poke at GC tasks starting next week

Syndicated 2010-05-10 06:07:34 from pudge

Perl 6 Design Minutes for 28 April 2010

The Perl 6 design team met by phone on 28 April 2010. Larry, Allison, Jerry, Will, and chromatic attended.

Larry:

  • caught up on a week's worth of backlog
  • made a few spec tweaks
  • discussed them with other people
  • trying to make error messages more awesome in STD
  • working on the ability to parse the insides of character classes
  • STD doesn't like parsing itself recursively there
  • need to iron out a few things
  • enum names can now be variables

Allison:

  • Debian packages ready to ship to Debian sponsors
  • putting together a list of GC tasks
  • cleaned out the existing page, have the big things listed
  • trying to decide which tasks to do first
  • doing a lot of reading and research
  • my little language project is due on Monday
  • HLLCompiler was enormously useful
  • will start working on the GC stuff next week
  • should also start a fresh pass through the ticket queue
  • added a workaround for the final remaining TT #389 bug
  • Jonathan had a test case

Will:

  • tried to focus on getting Rakudo blockers removed

c:

  • spent some time getting Rakudo to work with trunk
  • will need a Rakudo guts hacker for the last part
  • worked on the compact_string revamp branch with Vasily
  • merged now
  • that makes trunk about 12% faster than the 2.3.0 release
  • will work on a few Rakudo profiles once it works with trunk again
  • expect at least a 5% performance improvement there
  • have some other ideas, but won't do them without profiling first
  • came up with a scheme to reduce PBC size by coalescing strings
  • Peter Lobsinger is exploring that

Syndicated 2010-05-02 00:27:42 from pudge

474 older entries...

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!