Way(land) back when

It’s been a little while since I made progress on psadan ($dayjob and daddy duty), but from looking at the commit log I see it’s even longer since I wrote about any of it. So:

I wrote a macro. with-sync does what he name suggests: it executes its body, then sends a sync message to the wl_display, then waits for the callback. This is the standard Wayland way to make sure that stuff has finished happening before doing more stuff. The actual macro implementation was pretty similar to writing macros in CL except that the quasiquote escape is ~ not , – and that this is a Lisp-1, so declaring a local variable called promise turns out to be a bad idea when you also want to call the function (promise)

We refactored the handle-message and listen functions to return the new state of the agent that runs them, instead of keeping the globals in an atom inside it. Bcause we only update globals through the agent, this saves us from having to keep them in an atom, which felt kind of ugly.

And finally: psadan now parses the wl_drm protocol as well as the regular one – that bit was easy, it’s just another file of XML to parse. This is however where I got bogged down quite comprehensively in how we actually use the stuff though: there are a lot of bits. I’m hoping that JOGL , possibly in conjunction with its Pbuffer support, is (a) relevant, (b) sufficient, but even then there’s titting around with ioctls to be done which is not readily doable in the JVM.

Syndicated 2013-04-02 20:49:06 from diary at Telent Netowrks

In mysterious way(land)s

Why are our bind messages to wl_registry erroring out?

 
15:19 < daniels> yeah, wl_registry_bind is a special case - i think it's the 
                 one non-autogenerated marshaller we have
15:32 < daniels> actually no, sorry, i'm lying through my teeth
15:32 < daniels> it's no longer autogenerated
15:32 < daniels> but the parser expands new_id to interface + version + id
15:33 < daniels> it used to be hand-written, but is now autogenerated
15:33 < daniels> http://cgit.freedesktop.org/wayland/wayland/tree/src/scanner.c#n614
16:28 < jekstrand> dan_b: It is a somewhat-special case.  Basically, every time 
                   there's a new_id that does not have any type information 
                   specified, two aditional pieces of information (interface 
                   name and version) get added.
16:29 < jekstrand> dan_b: That really should be documented, but it's not.  I 
                   had to grep through the scanner.c file to find it all.

Armed with this additional info the fix was fairly straightforward: I augmented the XML parsing/protocol generation to stick an additional pair of arguments into each message that contains a new_id and no interface.

Thanks to both daniels and jekstrand on irc for clearing it up.

Syndicated 2013-03-21 13:29:35 from diary at Telent Netowrks

The long way(land) round

The latest round of psadan hacking was motivated by two goals

• that it would be good to actually remember the globals we’re receiving when we send get_registry to the magic registry object

• that if we’re going to ask for a callback when all the globals are notified, we should wait for it before continuing.

We went down a couple or three dead ends on our way to this goal, but eventually we settled on creating an agent responsible for listening to the connection (I called it channel, in the absence of any better ideas) and dispatching (using a handle-message multimethod) to some code appropriate for each kind of message we’re receiving.

Learnings

• multimethods the clojure way are pretty versatile: you can dispatch on pretty much any property – or derived property – of the function argument, not just on type. In our case that’s the interface name and the message (event) name:

   (defmulti handle-message 
     (fn [conn m] 
       [(:name (:interface m)) (:message m)]))
   
   (defmethod handle-message [:wl_registry :global] [conn m]
     (let [[name interface version] (:args m)]
       (conn/register-global conn name interface version)))
   
   (defmethod handle-message [:wl_callback :done] [conn m]
     (let [object (conn/get-object  conn (:object-id m))
  	 promise (:promise object)]
       (when promise
         (deliver promise m))))
   
   (defmethod handle-message :default [conn m]
     (println [“unknown message” (:name (:interface m)) (:message m)]))
  

• we handle the “tell me when you’re done” requirement with a promise. The get-globals code adds an unfulfilled promise to the callback object it creates, then once it has sent out its messages it derefs the promise , causing it to wait until something delivers the promise. That something is the handle-message implementation for wl_callback, for which, see above.

• we were trying to map handle-message onto each of the messages parsed out of the buffer, but not doing anything with the result. Given that map is lazy, this meant our handle-message code was for the most part not being called. Surrounding the map form with a dorun fixed this.

• you send work to an agent with (send fn ...) or (send-off fn ...) which invoke the fn with the current state of the agent, not with the agent itself. Which is fine but offers no facility for the agent to send work to itself – happily, the global/magic/special variable *agent*, which evaluates to the currently running agent if any is, works nicely for this purpose

  (defn listen [conn]
    (let [buf (conn/read-buffer conn)
          messages (buf/parse-messages buf conn :events)]
      (dorun (map #(handle-message conn %) messages)))
    (send-off agent listen)
    conn)
  (send-off channel listen)
  

• the ‘name’ field in a global is (confusingly) a number, and (more confusingly still) not an object id. Object number 3 in our client is a wl_callback object, whereas the global named 3 is .. well, let’s check …

  psadan.core> (def channel (chan/open-channel “/home/dan/private/wayland-0”))
  #’psadan.core/channel
  psadan.core> (chan/get-registry channel)
  ;; [debug output elided]
  {:object-id 3, :bytes 12, :interface {:index 2, :name :wl_callback, :requests (), :events ({:index 0, :name :done, :summary nil, :args ({:name “serial”, :type :uint, :interface nil})}), :enums ()}, :message :done, :args (-115)}
  psadan.core> (get @(:globals @channel) 3)
  {:interface :screenshooter, :version 1}
  

Next up? At some point we need to decide whether sending messages should be done by the channel or whether it’s OK to carry on doing that directly in whatever thread we happen to be in. But what would be much more fun is to see if we can actually render a window…

Syndicated 2013-03-18 21:07:47 from diary at Telent Netowrks

Finding a Way(land)

In the next round we shall be sending it the messages we have so lovingly composed from whole cloth and see if it reacts the same way as it did when the same bytes were sent from weston-info

And the answer is … yes, pretty much. We had to fix up our string parsing to make sense of the replies, but watch:

psadan.core> (def connection (conn/open-connection "/home/dan/private/wayland-0"))
#'psadan.core/connection
psadan.core> (defn test-send-message []
  (let [registry
        (conn/remember-object
         connection
         {:id 2 :interface (proto/find-interface-by-name :wl_registry)})
        done-cb
        (conn/remember-object
         connection
         {:id 3 :interface (proto/find-interface-by-name :wl_callback)})
        ]
    (conn/write-buffer connection
                       (buf/pack-message connection (:display connection)
                                         :requests :get_registry [registry]))
    (conn/write-buffer connection
                       (buf/pack-message connection (:display connection)
                                         :requests :sync [done-cb]))
    registry))

#'psadan.core/test-send-message
psadan.core> (test-send-message)
{:id 2, :interface {:index 1, :name :wl_registry, :requests ({:index 0, :name :bind, :summary "bind an object to the display", :args ({:name "name", :type :uint, :interface nil} {:name "id", :type :new_id, :interface nil})}), :events ({:index 0, :name :global, :summary "announce global object", :args ({:name "name", :type :uint, :interface nil} {:name "interface", :type :string, :interface nil} {:name "version", :type :uint, :interface nil})} {:index 1, :name :global_remove, :summary "announce removal of global object", :args ({:name "name", :type :uint, :interface nil})}), :enums ()}}
psadan.core> (def received (read-buffer connection))
#'psadan.core/received
psadan.core> (pprint (map (fn [x] [(:object-id x) (:message x) (:args x)]) (buf/parse-messages received connection :events)))
nil
([2 :global (1 "wl_display" 1)]
 [2 :global (2 "wl_compositor" 2)]
 [2 :global (3 "screenshooter" 1)]
 [2 :global (4 "text_cursor_position" 1)]
 [2 :global (5 "text_model_factory" 1)]
 [2 :global (6 "wl_data_device_manager" 1)]
 [2 :global (7 "wl_shm" 1)]
 [2 :global (8 "wl_seat" 1)]
 [2 :global (9 "input_method" 1)]
 [2 :global (10 "wl_output" 1)]
 [2 :global (11 "wl_drm" 1)]
 [2 :global (12 "wl_shell" 1)]
 [2 :global (13 "desktop_shell" 1)]
 [2 :global (14 "screensaver" 1)]
 [2 :global (15 "input_panel" 1)]
 [2 :global (16 "workspace_manager" 1)]
 [3 :done (58)]
 [1 :delete_id (3)])

My interpretation of what’s happening here is that we’re sending to the server a ‘tell object 2 about all your global objects’ message, followed by a ‘tell object 3 done when you’re finished doing stuff’ message, and as you can see from the output, the reply is a bunch of ids for global objects sent to object 2, a done event sent to object 3, and then a delete_id event for object 3 sent to object 1. I’m actually not sure why that last one triggers, as I don’t think I asked it to. Perhaps it’s just tidying up for me.

I’m also handwaving – if not actually handdrowning – a litle bit, because really … are these :global messages sent to object 2 or from object 2? For the moment, I am using the two directions interchangeably, which is probably not a recipe for an easier future life, but in the meantime I can continue to tread water.

It’s instructive, or at least reassuring, to compare this stuff with what weston-info says:

interface: 'wl_display', version: 1, name: 1
interface: 'wl_compositor', version: 2, name: 2
interface: 'screenshooter', version: 1, name: 3
interface: 'text_cursor_position', version: 1, name: 4
interface: 'text_model_factory', version: 1, name: 5
interface: 'wl_data_device_manager', version: 1, name: 6
interface: 'wl_shm', version: 1, name: 7
	formats: XRGB8888 ARGB8888
interface: 'wl_seat', version: 1, name: 8
	capabilities: pointer keyboard
interface: 'input_method', version: 1, name: 9
interface: 'wl_output', version: 1, name: 10
	x: 0, y: 0,
	physical_width: 1024 mm, physical_height: 640 mm,
	make: 'xwayland', model: 'none',
	subpixel_orientation: unknown, output_tranform: normal,
	mode:
		width: 1024 px, height: 640 px, refresh: 60 Hz,
		flags: current preferred
interface: 'wl_drm', version: 1, name: 11
interface: 'wl_shell', version: 1, name: 12
interface: 'desktop_shell', version: 1, name: 13
interface: 'screensaver', version: 1, name: 14
interface: 'input_panel', version: 1, name: 15
interface: 'workspace_manager', version: 1, name: 16

Top Wayland tip for today: it appears to be the case that you can make the C library clients log protocol exchanges to stderr by putting WAYLAND_DEBUG=client in the environment. When doing that it’s clear to see that weston-info is making a couple of additional requests that we’re not. We could add them, but I think the more pressing concern is to make it do something with the events we’re getting already – if it’s sending us details of global objects that we might need to know about, we should at a minimum be storing those details somewhere instead of throwing them away …

Syndicated 2013-03-14 13:32:01 from diary at Telent Netowrks

One more step along the Wayland

Yesterday’s lunchtime hacking was all about splitting the project into multiple files and getting it into git and onto Github – note that the mere fact of it being publically browsable does not imply that it will run, build, walk, make tea, perform any other useful function, or even forbear from exploding inside your computer and rendering the SSD to molten slag. Nor that I’m not still ashamed of it. It just keeps me slightly more honest.

Today I implemented enough of pack-message to be able to recreate the initial client→compositor message that we observed weston-info send last week. Still taking extraordinary liberties with signed vs unsigned longs, and plase note that all this code will work only on little-endian machines (there are any big-endian machines left?).

Lessons, puzzles

Leiningen does not need you to list the source files in your repository individually: it finds them magically. I believed otherwise for a while, but it turned out (slightly embarrassingly) that I had a parenthesis i the wrong place. My working hypothesis is that it assumes there is one namespace for each file, and any reference to a namespace it doesn’t know about it can be satisfied by loading a file with that name.

If I type (in-ns 'psadan.core) at the repl and that ns does not include a (:refer-clojure) form, I can’t use the symbols in clojure.core at the repl. I have not observed a similar issue wrt uses of clojure.core/foo in core.clj itself, just at the repl.

atoms! An atom is dead simple, really – conceptually at least, if not also under the hood: it’s a wrapper for an object that lets you look inside with deref and lets you change what’s inside with swap!. For each connection we use an atom holding a mapping from object ids to the corresponding objects, which starts out holding the singleton object for wl_display and then needs to be updated each time we generate an object locally and each time we learn of a new object from the peer.

(defn open-connection [name]
  (let [s (cx.ath.matthew.unix.UnixSocket. name)
        in (. s getInputStream)
        out (. s getOutputStream)
        wl-display (global-object-factory)
        ]
    {:socket s
     :input in
     :output out
     :display wl-display
     :objects (atom (assoc {} 1 wl-display))
     }))

(defn remember-object [conn id object]
  ;; (swap r fn args...) gets the current value of the atom inside r,
  ;; which for the sake of argument we shall call oldval, then sets the atom
  ;; to the result of calling (fn oldval args...)
  (swap! (:objects conn) assoc id object)
  object)

(defn get-object [conn id]
  ;; @foo is another way to write (deref foo)
  (let [o (get @(:objects conn) id)]
    o))

I have probably not chosen the fastest possible way of building up the messages I plan to send, in terms of fiddling around sticking vectors of bytes together. Will worry about that later if it turns out to be a bottleneck (but suggestions are welcome).

There was not a lot of Wayland learning this time. In the next round we shall be sending it the messages we have so lovingly composed from whole cloth and see if it reacts the same way as it did when the same bytes were sent from weston-info

Syndicated 2013-03-12 13:57:03 from diary at Telent Netowrks