Search Results for: glulx

More progress towards Hadean Lands on Steam

Here's a work-in-progress shot of Hadean Lands on MacOS. I'm using an extended version of Lectrote, with HL's map and journal windows added in. (The iOS release of HL has always had these, but not the Mac/Win releases. Until now!)

Yes, two different windows are titled "Map of the Marcher". I'll fix that.

(Background: Lectrote is a new interpreter for Glulx IF games -- meaning most recent Inform 7 games. It runs on Mac/Win/Linux, and it supports all Glulx features except audio. I still have a "beta" label on it, but it's been stable for people so I think it's about ready to 1.0-ify.)

Once this is ready, I'll soft-launch it as an update for existing HL users (people who bought the desktop version through Itch or Humble, plus Kickstarter backers). I'll also post the process of turning your Glulx game into a Lectrote app like this.

In other news, I was interviewed on another podcast! Guy Hasson of Blind Panels talks to me about pretty much the entire history of IF. Plus other stuff I've done.

Posted in Zarfplan | Tagged , , , , , , , | Leave a comment

Lectrote now autosaves; Hadean Lands is that much closer to Steam

I've been steadily updating Lectrote, my new cross-platform(*) IF interpreter. In the past month it's gotten icons, a preferences dialog with font and color options, and -- most exciting from my point of view -- autosave.

(* Cross-platform meaning that Lectrote runs on Mac, Windows, and Linux. The interpreter only runs Glulx games, not Z-machine or TADS or any other format.)

Autosave means that if you close the game window (or quit the interpreter) and then open it again, you will find your game where you left off. You don't have to use the SAVE or RESTORE commands unless you want to keep multiple save points.

As I wrote last month, autosave is a bit of a nuisance. I spent February getting it all polished up and tested. And then the tests revealed some obscure low-level bugs in the iOS implementation of autosave. Turns out my iOS Hadean Lands app was failing to store one VM table, and therefore running about 50% slower than it should have. Whoops. Good thing I wrote tests, right?

Lectrote on the desktop seems to be adequately speedy for most games, including Hadean Lands. So that's the last big technical barrier to creating a really nice HL app for Mac/Win/Linux...

I don't mean to imply that a Steam release is coming this week. It will still take some time to adapt Lectrote to a single-game interface. Naturally I will document this process! I want to make things as smooth as possible for any author who wants to release an Inform game as a Mac/Win/Linux app.

(The iOS process is, er, not very smooth. This is mostly because Apple's process for the iOS App Store is baroque, to say the least. I'm not planning to put HL in the MacOS App Store, so it should be simpler.)

I'll also see if I can include the extra dynamically-updating windows from the iOS version of HL: the clickable map and the alchemy index. In theory, these aren't too hard to set up -- I can copy the logic and contents right over from the iOS app. In practice, theory sits on the curb and laughs at you when you say things like that. So we'll see.

But the end is in sight. Give me another couple of months.

Once I have a working HL app, I will release it as an update for the existing Mac/Win/Linux versions of the game. If you have downloaded HL from the Humble Store or Itch.IO (either as a purchaser or a Kickstarter backer), you will be able to download the new app and try it out. If no horrible bugs turn up, I'll start preparing the Steam release.

Posted in Zarfplan | Tagged , , , , , , | Leave a comment

Introducing Lectrote, an interpreter

Today I posted the beta of Lectrote, a new IF interpreter application for Mac, Windows, and Linux. This is both more and less exciting than it sounds!

If you're familiar with the IF scene, you know that there are several applications which can be used to play these games. Zoom (Mac), WinGlulxe (Windows), and Gargoyle (multiplatform) are the most commonly used. And then there's Quixe, which is a Javascript-based interpreter used on iplayif.com and other web sites.

When I was looking to release Hadean Lands as an app, I found that none of these were really what I wanted. Zoom is unmaintained and buggy; WinGlulxe is weird about scrolling; Gargoyle has problems on hi-res displays. (I'm summarizing, it was a long messy story.)

Quixe had the UI that I wanted -- no surprise; it's the one I wrote the UI for! -- but it wasn't really meant to be used as an app. It exists as a web page, or a component of a web page. Also, it's slow. So I put it aside and went with Gargoyle.

However, the long messy story didn't end there! A few weeks ago I was gazing over the endless cycle of dev-tools and noticed Electron. Electron lets you wrap up a Node.js tool as a standalone app for Mac, Win, and Linux. And Node.js is, well, I don't really know what it is but it's a web thing. Seems ideal, right? Stuff Quixe's web page into Electron and we're done.

It wasn't quite that easy. Node.js has full filesystem access (unlike a web page), so I had to extend Quixe's load/save system to deal with ordinary files. (So you can exchange save files between Lectrote and other interpreters.) But that was still pretty easy. I stuck the IF postcard in a menu, too.

And now you can try it.

So what does this have to do with getting Hadean Lands onto Steam? Well, it's a very simple tweak to drop a Glulx game file into Lectrote. Then you've got a Mac/Win/Linux app that plays a single game. And it looks nice and the text layout is pretty and you can adjust the font size without editing a text file.

I haven't done that yet. I'll have to adjust the menus -- knock out all the support for opening multiple games.

More important, I'll have to add autosave. Right now, if you're playing a game and you close the window, your game is gone. Hope you typed SAVE! That's okay for an interpreter (used by IF habitu├ęs), but it's not ideal. It's really not acceptable for a Steam standalone game release.

Autosave for Glulx games is a bit of a nuisance, but I got it working on iOS. I will get it to work with Quixe. It will just take a few more weeks.

...oh, and then there's the speed. I mentioned that Quixe is slow, right? It's faster than it was but it might not be fast enough for Hadean Lands. If you own HL for Mac/Win/Linux, try it! In particular, try loading a mid-game save file and typing a command which requires many stages, like GO TO BAROSY.

(If you don't own HL, may I remind you that it's on sale for the next two days? I probably don't have to. But I do it anyway.)

Anyway, I may try plugging a different Glulx VM into Lectrote to speed it up. I can probably run RemGlk/Glulxe as a subprocess of the Node.js server... We'll see.

For now, Lectrote is a multi-platform interpreter app which has the UI I want, and that's a good start.

Posted in Zarfplan | Tagged , , , , , , | 8 Comments

Customizing an interpreter for a Glulx game release

Another technical question from Twitter: the integration of Hadean Lands with its iOS app. How did I set up iOS UI features like the dynamic map and the recipe index?

(Warning: if you don't think C code is interesting, this post is not for you! Sorry.)

The iOS version of HL uses my standard iOS IF interface, extended. I've added two tabs to it. The map tab shows your current location, and you can tap to travel to any room you've visited before. The recipe tab shows an index of recipes and other information you've learned. These work just like the "GO TO..." and "RECALL..." commands, so they don't make the game easier to solve, but they're convenient shortcuts.

I'm not going to post the iOS UI code I used. If you know iOS programming, it's very basic -- textbook UITableView and UIImageView stuff. Instead, I'll talk about the general problem: transferring information between the Glulx VM and your native (C) interpreter.

I should put "general problem" in quotes. There are several Glulx interpreters, after all. But let's assume that you're building a native app for your Glulx game, incorporating the glulxe interpreter engine (in C), and you want to customize it with game-specific features. You've implemented the UI part; now you just need to extract game state information. Say, the player's location to show on the map.

There are a couple of approaches that would work. For example, we could define a completely general system for transmitting game information to an outside observer, and add that to the Glk spec. Sound like a good idea? Well, maybe, but it's both a hard problem and a vague problem -- what's "information"? We'd probably need some kind of structured interchange format (XML? JSON?). Then we'd have to encode and decode that. Plenty of headaches. No thanks, not right now.

Or we could define a new Glk output capability just for this game. No spec, just define a function ("glk_set_map_location") and have the game call it each turn. I thought about this, but I decided it would require modifying too many different modules. Glk function dispatching is kind of ugly.

Instead, I decided to write C code to peer directly into Glulx VM memory! It's stored as a simple byte array, after all. Reading the game state out is just a matter of understanding the memory layouts of objects, variables, and arrays.

Okay, that's not easy, but it's doable in a small amount of code. To make this work:

  • You'll need to know some Inform 6. Sorry. Inform 7 is a wonderful high-level programming system, but it compiles into low-level objects, variables, and arrays.

  • You'll need to extract information from the gameinfo.dbg file that's built along with your game. (This lives in the Build subdirectory of your Game.inform project.) This is where you find the memory addresses of those objects, variables, and arrays.

  • You'll need some boilerplate C code to examine objects, variables, and arrays. I'll attach it to this post; you can copy-and-paste.

How do you use this? Say you're interested in the player's location. (For the map, right?) You know from the Standard Rules that

The location variable translates into I6 as "real_location".

Browse through gameinfo.dbg with a text editor (or an XML editor) and you'll see a stanza that starts:

<global-variable>
    <identifier>real_location</identifier>
    <address> 249528</address>
    [...more info...]
</global-variable>

This tells you the absolute memory address of the I6 variable real_location. Call gameparse_get_global() with that address -- the function is shown below -- and you'll get the player's location, expressed as the memory address of the room object.

How do you know what room that is? Somewhere in gameinfo.dbg is another stanza:

<object>
    <identifier>I88_kitchen</identifier>
    <value>     273861</value>
    [...more info...]
</object>

This indicates that an object named "Kitchen" has memory address 273861. (It happens to be the 88th item that the compiler defined.)

Obviously this is not very convenient. All of these addresses are liable to change every time you compile your Inform game. So you wind up writing a script to parse your gameinfo.dbg XML, locate the real_location address, locate an object called I#_kitchen (for some integer, might not be 88), and so on. I like to write them out to a C header file looking like this:

#define GLOBAL_REAL_LOCATION (249528)
#define OBJ_KITCHEN (273861)

Then you can include this header in your game project and write code like

if (gameparse_get_global(GLOBAL_REAL_LOCATION) == OBJ_KITCHEN) {...}

But you'll have to write this XML-parsing script yourself, I'm afraid. I don't have one for you. You could start with profile-analyze.py.

Note that you can't do this for I7 global variables (declarations like "Foo is a thing that varies.") Those get tucked into an I6 array and the gameinfo.dbg file has no information about them. If you need to observe a global variable, you'll have to declare it in I6 with an I7 translation. See I7 manual 27.22.

The gameparse_obj_child, gameparse_obj_sibling, gameparse_obj_parent functions let you traverse the I6 object tree. (Although this doesn't give you I7 relations like component-ness.) I used this for HL's recipe index. The game's internal knowledge objects are moved into special containers, for easy scopability, so I can trawl those containers to set up the recipe tab.

Looking at object attribute and properties is easy. The functions gameparse_obj_attribute and gameparse_obj_property do that. Knowing what I6 attribute or property represents a given Inform property is harder. The best plan is to to look at the generated I6 code (the Build/auto.inf in your Inform project) and see what's going on under the covers. For examine, the boolean property open/closed is implemented by the following I6 lines:

! meaning of "open"
if (t_0) return (GetEitherOrProperty(t_0, open));

This means you're looking for an I6 attribute in the gameinfo.dbg file:

<attribute>
    <identifier>open</identifier>
    <value>13</value>
</attribute>

Or say you've written the I7 line:

A thing has a number called the weight.

You find that this generates I6 code like:

! [1: let n be the weight of the noun]
tmp_0 = GProperty(OBJECT_TY, noun, p15_weight);

And so you're looking for an XML stanza like:

<property>
    <identifier>p15_weight</identifier>
    <value>268</value>
</property>

At this point you start to wonder if the general information API wouldn't be better after all. Maybe it is. Why didn't I go that way? (I realize this is more of a peek into Zarfbrain than you probably care about.) I find that this stuff is the easy part. Drawing a map was hard. Extracting recipes from game output and importing them into an iOS app was work, if not really hard. Extracting state from VM memory was a solved problem; I just had to do it for a lot of objects.

Also, it's fast. I only inspect game state when flipping to the relevant tab, and the inspection code is C. If I'd rigged the game to output state to an API, it would have to be every turn, which would slow down normal gameplay. Or I guess I could have added a secret input event for tab-flipping, which would mean blocking the UI on game code, also slow... Anyhow.

Some other concerns:

  • In my iOS interpreter, the VM runs in a background thread. The UI runs in the main thread, as is usual for iOS apps. How did I synchronize the VM inspection? I didn't! Totally whiffed the thread-safety issue. It's generally not a problem; the player will flip tabs while the VM is blocked awaiting input. But if you have some fancy timed-input code that moves the player around, and the player flips tabs at just the moment when the timer fires, you could get a bum value out of real_location.

  • What about Z-code games? You can take the same approach, but you need different state-inspection code, because the Z-machine's memory layout is different. (For a start, it's all 16-bit words, not 32-bit.) I did some of this for the iOS releases of Dreamhold, Shade, and Heliopause. But I didn't wrap up the C code into nice functions like these. So -- exercise for the reader, sorry.

  • Can you do this in Quixe? (A Javascript interpreter instead of a C interpreter.) Yes, and the plan is just about identical. Quixe maintains a private memmap array, which is a Javascript array of byte values, so you just have to translate the code below into Javascript and you'll get the same results. The only trick is that memmap is in a private scope. You could add the functions below to the Quixe global object, or rely on ReadWord/ReadByte, or just add a one-liner to export memmap:

    get_memmap: function() { return memmap; },
    

Finally, we have the question of input. When the player taps a room on the HL map, the game must accept it as input.

Again, there are a few ways to handle this. I decided to use a custom Glk event. The Glk spec says that negative event ids (0x80000000 to 0xFFFFFFFF) are reserved for implementation-specific events, and that's what this is. The iOSGlk library has a forceCustomEvent method. The map UI invokes this, passing a negative constant as the event type and the room object address as an extra argument. Conveniently we've already extracted the addresses of all the rooms from gameinfo.dbg.

(Other Glk libraries might not have this sort of API, but it will be easy to add. All events funnel into the Glk library in the same way.)

The only remaining chore is for the game to react to this custom event. Unfortunately, Inform's core parser loop is built to accept only text. This will have to be improved someday! (Not just for custom map hacks, but for hyperlink input, mouse input, and so on.)

But, again, I took the cheap way out. I used the HandleGlkEvent hook to translate the custom event into the input line "MAP-VERB 273861", where the number is the decimal room address.

This is not ideal because a player could type that line by hand! Well, whatever. Players can do what they want. Mind you, I sanity-check the argument very carefully to make sure an invalid address can't crash the game or leave the player stuck in a teakettle.

The relevant I7 code (and I6 inclusions) appear at the end of this post. I just noticed, though: it won't compile with the current Inform 6 compiler (6.33)! This is because I rely on the constants #lowest_object_number, #highest_object_number, #highest_class_number. These were missing from the Glulx compiler until, well, until I was writing this map code and needed them. You can build the latest I6 compiler from source and shove that into your I7 distribution. Then this'll work.

So the conclusion is, this is all a big pain in the butt, isn't it. Yep.

(No comments out of you, Dave. It really was the least-effort solution for my particular problem.)


C code for examining Glulx VM state:

#include "glulxe.h"

/* The glulxe.h header defines the glui32 type and the Mem4() macro.
    Also the memmap global variable (array of bytes). */
/* These functions do a little bit of safety-checking, but you really
    should be careful to only call them with valid object addresses. */
/* Feel free to add warnings or errors to the "error" cases. I like to
    throw exceptions, myself. */
/* All of this code assumes that NUM_ATTR_BYTES is 7, the default
    value for Glulx games. If you increase NUM_ATTR_BYTES, you'll
    have to adjust the object structure offsets. */

int gameparse_mem_active(void)
{
    return (memmap != NULL);
}

/* Fetch a global variable. */
glui32 gameparse_get_global(glui32 addr)
{
    if (!memmap)
        return 0; // error: called get_global with no memory map
    return Mem4(addr);
}

/* Get the object which contains a given object, or 0 if it
    is off-stage. (The I6 parent() function.) */
glui32 gameparse_obj_parent(glui32 obj)
{
    if (!memmap)
        return 0; // error: called obj_parent with no memory map
    if (memmap[obj] != 0x70)
        return 0; // error: called obj_parent on a non-object
    return Mem4(obj+5*4);
}

/* Get the first object contained by a given object, or 0 if it
    has no contents. (The I6 child() function.) */
glui32 gameparse_obj_child(glui32 obj)
{
    if (!memmap)
        return 0; // error: called obj_child with no memory map
    if (memmap[obj] != 0x70)
        return 0; // error: called obj_child on a non-object
    return Mem4(obj+7*4);
}

/* Get the next object contained after a given object, or 0 if there
    are no more. (The I6 sibling() function.) */
glui32 gameparse_obj_sibling(glui32 obj)
{
    if (!memmap)
        return 0; // error: called obj_sibling with no memory map
    if (memmap[obj] != 0x70)
        return 0; // error: called obj_sibling on a non-object
    return Mem4(obj+6*4);
}

/* Look up an attribute flag on an object. */
int gameparse_obj_attribute(glui32 obj, int attr)
{
    if (!memmap)
        return 0; // error: called obj_attribute with no memory map
    if (memmap[obj] != 0x70)
        return 0; // error: called obj_attribute on a non-object

    unsigned char byte = memmap[obj+1+(attr>>3)];
    if (byte & (1 << (attr & 7)))
        return 1;
    else
        return 0;
}

/* Look up a property value on an object. */
/* Returns the first word of the property, if multi-word. (In most I7 games,
    the only multi-word property is "name". So you can't use this function
    to scan through the name list of an object.)
    If the property is not provided for this object, returns 0. */
glui32 gameparse_obj_property(glui32 obj, int prop)
{
    if (!memmap)
        return 0; // error: called obj_property with no memory map
    if (memmap[obj] != 0x70)
        return 0; // error: called obj_property on a non-object

    glui32 proptab = Mem4(obj+16);
    glui32 propcount = Mem4(proptab+0);
    for (int ix=0; ix<propcount; ix++) {
        glui32 propent = proptab+4+ix*10;
        int pid = Mem2(propent+0);
        if (pid == prop) {
            glui32 paddr = Mem4(propent+4);
            return Mem4(paddr);
        }
    }

    /* Property not provided. */
    return 0;
}

Inform 7 code for accepting custom input events. (This assumes a "select one room on a map" action, but it can be adapted to other uses.)

Include (-
[ HandleGlkEvent ev ischar args   val;
    if (ischar == 0 && ev-->0 == CUSTOM_EVENT_ID) {
        val = ev-->2;  ! the object address of the tapped room
        glk_cancel_line_event(gg_mainwin, gg_event);
        ! Write a synthetic "MAP-VERB ###" command into the buffer.
        VM_PrintToBuffer(buffer, INPUT_BUFFER_LEN-WORDSIZE, PrintVisitNum, val);
        return 2;
    }
    return 0;
];
[ PrintVisitNum val;
    print "MAP-VERB ", val;
];
-) before "Stubs" in "Glulx.i6t".

To decide what object is paranoid-object-check (N - number): (- ParanoidObjCheck({N}) -).
Include (-
! Return val if val is a valid object, otherwise 0.
! This code requires the bleeding-edge (6.34) Inform 6 compiler.
[ ParanoidObjCheck val;
    if (val < Class + ((#lowest_object_number + #highest_class_number) * GOBJ_TOTAL_LENGTH))
        return 0;
    if (val > Class + ((#highest_object_number) * GOBJ_TOTAL_LENGTH))
        return 0;
    if ((val - Class) % GOBJ_TOTAL_LENGTH ~= 0)
        return 0;
    if (val->0 ~= $70)
        return 0;
    return val;
];
-).

Numeric-visiting is an action applying to one number.
Understand "map-verb [number]" as numeric-visiting.

Carry out numeric-visiting:
    let N be the number understood;
    let O be paranoid-object-check N;
    if O is nothing:
        instead say "That address ([N]) is not an object!";
    if O is not a room:
        instead say "That address ([N]) is not a room!";
    [Check locked doors and so on here...]
    now the player is in O;
    say "You move to [O]."
Tagged , , , , , | Leave a comment

Forward planning for IF tools

I've been bemoaning the slightly run-down state of IF interpreter software. (The confusing font preference system in Gargoyle is just one example.) The fact is that the big surge of open-source IF activity was the late 90s and early 00s. Since then, coders have been drifting out of the community, and the ones still around have gotten lazy. (I include myself in that indictment, for damn sure.)

As a community, we do not have a tradition of mentoring and fostering new contributors to IF projects. All of our projects were made by people (most often solo developers) who got excited and wrote a whole application or library.

I like to think that we've got a good software stack, which smooths the path a little. You can write an Inform extension or a Glk library port or a Glulx engine core or a Parchment web service, and it will fit into the ecosystem. But it still starts with a person showing up with enough energy to start, build, and finish an idea. If someone shows up who is curious but not committed, we nod companionably and wait to see what happens. The results, over time, are predictable: activity slows down and stops.

With that introduction, you'd expect me to go on and talk about mentoring. But I don't know anything about mentoring. I'm one of the control freaks! I'd rather work on my own projects than collaborate.

Anybody want to think about community-building? (Hopeful look around...)

(Of course a lot of my projects are specifications and tools that interoperate with other people's code. So I kick myself in the ass and make it happen. But my natural talents do not lie in management.)

In this post, I'm going to talk about my plans as a solo IF tool developer. Warning: I will also talk about money.


I have a whole list of things I think Need Doing In IF (see footnote) but the one I intend to tackle next is Glk 2.0. This is pretty much the same Glk 2.0 plan I've been planning for years, but since it's been years, not everybody may remember it. (Or perhaps you remember it with bitterness and loathing, in which case I apologize.) In any case, I will explain it from the top.

The plan is: integrate Glk (the Glulx VM display layer) with CSS and Javascript. That is, you will be able to distribute Inform games with custom stylesheets and special display effects (see Vorple).

Why CSS and Javascript? Because they have already conquered the universe, so we might as well give in to our anger and rule at their side. Also, the most commonly-used IF interpreters are written in Javascript, which makes adding these new features trivial. (If we plan correctly.)

So how do we approach this? I'll admit up front that this plan isn't complete. This is a sketch; I have parts of it worked out.

The basic idea is to define a canonical HTML representation of Glk output. Or rather, a canonical DOM representation. (The DOM is the internal data structure that a browser builds when it reads in an HTML file.)

Currently, the Glk spec describes output in terms of windows, text, and text styles. The new spec will explain this in terms of HTML <div> and <span> elements, and the CSS classes for each one.

I don't mean that your Inform code will output HTML. For simplicity, and to maintain a clean separation of concerns, the Glk calls in the Inform library will remain the same. You'll set a style and print some text. The interpreter's job will be to either translate those calls into HTML (as defined by the spec), or do the equivalent job of text display.

Nor do I mean that interpreters will have to support every crazy CSS trick that Firefox is capable of! Far from it. The idea is this: whatever display capabilities the interpreter does have, let CSS guide where it's used.

It's easy to misunderstand this. Let me lay it out in an organized way.

  • This is a change to the way the interpreter (Glk library) manages the display, not a change to the way you write your Inform game.
    • But there will be some new capabilities available for Inform games. Mostly this will be about letting you use more styles.
  • The display of a stock Inform game will not change much. Any changes will be in the direction of more uniformity between interpreters.
  • This should not require interpreter maintainers to rewrite their code from scratch.
    • Javascript interpreters (Quixe) will work by generating HTML information and shoving it into the browser display. This is how Quixe has always worked. I'll just have to add the CSS and make some adjustments.
    • C interpreters (Gargoyle, WinGlulxe, Zoom, Floyd-bot, etc) can continue to work the way they work, by calling OS-native text-display calls (or use SDL or telnet or whatever). However, to decide on text display attributes for each style, they should start using CSS information. I will provide a library to make this easy.
    • Alternatively, C interpreters could switch over to HTML display. That is, they could throw up an OS-native HTML widget and shove HTML into it, just as Quixe does. This is optional, but it has some advantages -- see below.
  • The new plan will initially be available through an I7 extension. I expect that some future I7 release will make it the default.
  • What new features does this plan give us?
    • A game author will be able to include a CSS stylesheet with the game. This will customize the way the interpreter displays the game.
    • A player will be able to select a CSS stylesheet as an interpreter preference. This will customize the way the interpreter displays the game (overriding the author's stylesheet if necessary).
    • There will be some way to display more and fancier styles. I know we had an argument about this a few years back; let's not restart it in this post. I haven't nailed down the details anyhow.
    • A game author will be able to include a Javascript library (or libraries) with the game.
      • For HTML-based interpreters, the Javascript will be loaded into the browser page (or HTML widget). It can then perform arbitrary display tricks. Glulx Vorple can be built on top of this.
      • For interpreters which do not have an HTML layer (i.e. old-style C interpreters), Javascript tricks will not be possible. The game will have to degrade gracefully or give up.

Where am I with this plan? Not very far. Even after all these years.

I have a roadmap. I've just finished step 1. It will be a few steps more before anybody else has to start writing code.

  1. Write a simple, dependency-free C library to parse CSS. Done! I thought this would be a weekend hack; it actually took many days of work, because CSS is just as horrible as every other Web technology. (No, I didn't search for prior art before I started hacking. Who do you think I am?)

  2. Nail down the HTML representation and the Glk API changes it will require. Think about how it affects all existing interpreters.

  3. Update GlkTerm to the new model. GlkTerm is a terminal-window library (it's built on ncurses), so it can only support a few text attributes -- boldface, underline, ANSI color. This is enough for a proof of concept.

  4. Update Quixe to the new model. (And jQuery, while I'm at it.) Quixe is an HTML-based interpreter so it will allow the full range of CSS and Javascript fireworks.

  5. Write the I7 extension that enables this stuff.

  6. Write a lot of demos. (Not just to show off! Demos will let other interpreter authors know they're hitting the mark, display-wise.)

  7. Update RemGlk and CheapGlk.

  8. Unleash my reign of terror upon the world.

Easy, right?


You can reasonably ask how long all of this will take. Especially since I laid out much the same plan at least five years ago, and then again three years ago, and I've just managed to finished step 1.

Well, Hadean Lands is done. That's a big carcass off my plate. Also, for this semester I'm spending one day a week at MIT(*) and that's a nice regular schedule for working on the project.

(* But not this week, because it snowed some. And the Red Line caught fire or something.)

It's still a lot to promise; enough that I feel queasy just posting this post. I hate making promises that I don't keep.

Thus to the subject of money.

Post-HL, I would like to work more on open-source IF tools. I would also like to work on indie games that will pay the rent. (HL does not pay the rent in the long term, despite the stellar reviews I got this month.) There's also contract work and so on. Life balance, it's hard.

One way to converge these interests: find a way to crowd-source IF tool work. But this is not an easy model. It's not a Kickstarter -- it's ongoing work, not one big deliverable. It's not exactly a Patreon, because I can't give backers early access or bonuses. (But some people try this anyway?)

I'm signed up for Gratipay, which started as a tip model for GitHub users (but now supports any developer). I haven't publicized it at all; despite this, somebody is giving me 33 cents a week. (Thanks!)

Are there any other patronage systems which you folks think I should investigate?

(Note: I am not asking for money at this time! I'm thinking about ways in which I might ask for money in the future.)

I realize this is an awkward subject. Many people support IF software. I don't want to be The One Who Gets Paid while everybody else labors away for free. On the other hand, there's a lot of work that isn't getting done; this has been true for years. I can start to unstick this, but not to the exclusion of paying work.

Furthermore, the Glk 2.0 plan isn't the only thing on my plate. I'd love to get back to Tworld (the Seltani engine). I could dig into CocoaGlk and figure out why Mac Zoom occasionally crashes (and why Mac Inform 7 occasionally trashes my skein, and maybe why it occasionally eats somebody's source code...) I could spend some time experimenting with the rule-based language idea, zog help me.

What I'd really like is a patronage-and-voting system, where people could put money into different projects, with the understanding that the ratios determine how much time I put into each one. Does anything like that exist? (No I'm not going to build one.)

...Wow, this post has wandered in circles, hasn't it. A mix of plans, promises, positions, and pleas!

I guess I'm trying to figure out where the next phase of my life falls, relative to the IF community. I have taken stock, and this is what I see. Comments welcome.

Posted in Zarfplan | Tagged , , , , , , , | 29 Comments

Zarfplan: February was kind of nuts

I suppose you want the room count, don't you. I will give it to you. When? Now, I should think. You look forward to it, do you? I think you do. ...Nine rooms, I got done this month. All clues, all descriptions, all scenery, all actions handled. I can play through those nine rooms and it is a solid, playable IF game.

This is most of what I call "chapter 0" in my notes. After another couple of puzzles, the player transitions to "chapter 1". That's where most of the game opens up, at least a little bit.

But how many rooms are there in this game? I hear you cry. Okay, that's a fair question. There are 82. So you might conclude that I am 11% done with this thing and have eight months of work remaining. And that's... not a completely terrible estimate.

I don't mean that it's my estimate. Or that it's accurate. Rather, I guess, that the possible errors fall in both directions. So I can't say it's definitely too long or too short.

Not all rooms are created equal. Laboratories and storage rooms are packed with exciting objects. Hallways are fairly boring. (They have just a couple of exciting objects that I moved from the storage rooms to even things out.) Maybe 25% of the rooms are hallway. They go by quickly.

Some of this month was spent constructing scenery classes which I will reuse throughout the game. That will save time later. On the other hand, I didn't have a chance to start the tutorial, which will run throughout the actions in the first room and then mostly drop out. A tutorial is hard work and may chew up a couple of weeks of March on its own.

I still haven't tackled the "monocle" plan (changing how some bits of paper are presented), nor one of the environmental spell components. (Neither appears until "chapter 1".) There's some planning to do there.

Some of this month was spent profiling the HL code, just to see if there were any horrible inefficiencies. Of course, there were. So I dug into a little low-level hackwork to speed up certain I7 activities. I will have to do more of this in the future.

So I add all those factors up and shrug. I got a lot of implementation done in February. It was a good month.

What else happened in February? I worked some on Project Edifice Gumshoe. I got as far as an iPad prototype, I showed it to people... total flop. The puzzle mechanic is simply opaque to players. Yay prototyping! I have some ideas for starting over. Boo starting over! Depressing even when it's necessary. But the code from the prototype will be recycled into something. I got clever with iOS animation transitions, and the result is very tactile.

I helped out with an effort to update the Z-machine specification documents. This is not strictly in my wheelhouse -- Hadean Lands runs on Glulx, my successor to the Z-machine. But David Fillmore decided to clean up a bunch of unfinished editing and error-correction from the old days... by which I mean 1997... and that turned into a bunch of online discussion. Plus I have the keys to the web site. So I put in some time on that.

There was also a proposal about adding text captions to IF cover art, and in general to images in IF data files. (For accessibility, like alt tags on web images.) This is in my wheelhouse; I proposed the IF data file spec so I get to organize updates to it. Then we looked at the IF Babel site (covering bibliographic information, including cover art) and realized that nobody was in charge of it; it's just been sitting untouched on the Archive since 2006. So I cranked up a mailing list and got that side moving. (The Blorb side is awaiting comment; I'll add it to the site next week.)

And today I dug into a completely tedious idea about adding more boolean flags to the Inform 6 object structure... which I thought I implemented in '98... but it turned out I forgot. Result: a completely tedious patch to the I6 compiler, which works great. (Unit test!) Only my clever optimization idea of '06 didn't take this into account, so it breaks when you run it. I'll have to update the Glulx spec too.

Will Hadean Lands benefit from more boolean flags? Probably not! Because even if I get the interpreter fixed, I7 doesn't know to take advantage of it! Yes, IF is still software engineering: messy. But I move my parts forward one step at a time.

And I played the new RealMyst update, and some of the Dishonored DLC from last year. Gotta finish that this weekend.

Posted in Zarfplan | Tagged , , , , , , | Leave a comment

The techy post, part 4 of 4

So, here's the Glk and Glulx update plans for 2011.

CSS stylesheets:

One of the big gaps in my original Glk spec was in specifying fonts, font sizes, and other text display parameters. On the left hand, I wanted this to be a player preference most of the time. On the right hand, IF authors want to provide a nice customized design for their games. I came up with what I thought was a minimal compromise, but it wound up almost useless from the game author's point of view. (You can't specify a font at all, for example; font sizes are poorly specified and therefore inconsistently implemented; and so on. When I wrote Quixe, I skipped the font-style features entirely, because they weren't worth the effort.)

This has been on my "improve please" list for several years. However, the left-hand-right-hand tension I've described still exists. You see the same thing in web design and e-book design; each of these has reached its own compromise between author preferences and audience preferences.

I'm still not sure I know where the balance should be. (After several contentious discussions in the IF forums, nobody else is sure that I know either.) However, I know where I want to start, which is to let the author include a CSS stylesheet. CSS is the worst formatting system HTML could have used except for all the others, and that's what I want for IF too.

People are likely to already know some CSS, at least.

(You can already use a CSS stylesheet with Quixe; it is a web app, after all. Just edit the included CSS file. But I need to specify this in a way that all interpreters can approximate -- even those that don't have CSS engines available.)

Quixe graphics and sound:

The Glk spec has graphics and sound options, but I haven't implemented them. It just wasn't a first-release milestone. Many IF games use Glulx solely for the extra memory space, but some (e.g. Emily Short's Alabaster) make good use of graphics, and there's no reason a web browser can't handle them.

There are some reasons that it's a hassle, though -- having to do with the way we package IF games and graphics data.

Quixe optimizations:

A simple request. We're talking about an IF VM interpreter written in Javascript, running in a Javascript interpreter written (probably) in C. That's a slow stack of code. Quixe is inherently slower than Parchment, because it deals with more data -- that's the point of Glulx, and using 32-bit values everywhere doesn't help.

Also, Inform 7 isn't as efficient about its compiled code as it could be. Real CPUs have surged way ahead of IF complexity in the past thirty years, and that gave Inform room to be, well, a little bit slack. Nobody noticed -- as long as they were running native-code interpreters on modern machines. But when we started porting interpreters to cell phones, and web browsers, and microwave ovens, then the inefficiencies became visible.

Both Parchment and Quixe use a just-in-time compilation strategy. Chunks of VM byte-code are compiled to Javascript as they're encountered. Modern web browsers can compile Javascript to native code as it's encountered. So we get much better speed than we would have a few years ago, before the Javascript arms race started. But Quixe could be smarter about its compilation strategy, in several ways.

Javascript features:

Since Quixe is a web app, couldn't we forget about specifying a graphics API for Glk, and just let the game fire Javascript events? The author could put arbitrary HTML on the game -- even Canvas or SVG displays -- and then manipulate them directly.

I want to approach this cautiously. It can't just be a free-for-all of manipulating Quixe internals; that would be a horrible quagmire in the long run. And we do want other interpreters to support these features (where possible).

Glk already has the notion of a rectangular image pane inside the game UI. If we extend that to allow HTML/Canvas/SVG panes, then most interpreters will be able to support them by embedding WebKit or other HTML widgets. So this is possible; but again, there are hassles of packaging to work out.

Accessibility:

If you watched Get Lamp you heard from several blind or visually-impaired IF fans. I've always wanted my IF interpreters to be accessible, but I don't know much about the mechanics of screen readers -- I've just assumed that it could be made to work.

We've had some discussion on the topic in the past few months; see this blog post, for example, and this forum thread. I need to go over Quixe and figure out what work it needs.

This will also apply to the iPhone app, too. (If you have not yet, you should read Austin Seraphin's post about getting an iPhone. He was in Get Lamp, the guy talking about not needing a lamp.)

Network support:

This comes up in two distinct use-cases. First, the ever-sounding siren song of multiplayer IF. This has been tried in various ways (all the way back to MUDs, and more recently Guncho, which is a MUD-like assemblage of Inform 7). But if the game itself had the ability to open (and listen for) network connections, we might get some new angles. (Perhaps go above the TCP layer, and leverage some existing social or gaming network?)

Second, access to web services. Could an IF game usefully query Wikipedia, Wordnet, or your local weather for in-game effect? Maybe.

Assorted random features:

An IF game should be able to read the system clock -- either to display a real-time clock, or measure how long the player has been doing various things.

An IF game should be able to normalize Unicode lines of input text. (If you know what Normalization Form C is, well, we're currently not doing it.)

There are some restrictions on line input that I need to loosen. (These have to do with whether the command line can be cancelled or edited by the game as the user types. It's not a feature of most games, but occasionally it's desirable.)

A way to define whether display panes in the game UI are separated by borders. I've only been promising that for about ten years now. (This ought to be handled by CSS, but boring technical reasons may prevent that.)

And...?

I could be doing plenty more, beyond those ideas. Brand-new fully-modern virtual machine? Fill out the rule-based programming language that I've been thinking about since before Inform 7 was released? Finish the kung-fu ninja game?

I'm trying to keep some boundaries in place, though. I intend to sleep occasionally. Maybe go outside once a month.

Posted in Zarfplan | Tagged , , , , , | Leave a comment

The techy post, part 3 of 4

The current state of the Inform and Glulx universe...

Present and Future

The next major change in the IF landscape was in 2006, when Graham Nelson released Inform 7, a brand-new high-level IF design system. I7 was (and is) still compatible with the Z-machine... but its very existence put a much greater strain on the system. A simple one-room Inform 6 game might be 64k; the same game in Inform 7 was over 128k, simply because of the more abstract and powerful library model. (In the most recent version of I7, such a game is 192k.)

The good news was, I7 was compatible with Glulx almost from the beginning. Nobody had any doubts about the fate of the Z-machine at that point.

(As it happens, I7 uses I6 as an internal compiler stage. The I7 compiler generates I6 code, which then I6 compiler then converts into either Z-code or Glulx game code. If that sounds horrific, look up the history of C++.)

The major change after that was Parchment, in 2008. This was an open-source Z-machine interpreter, just like dozens going back to the 1990s -- except Parchment was written in pure client-side Javascript. You could open an IF web page in an ordinary browser, and be playing the game immediately. In the web era, this is an absolute necessity.

(Parchment was not the first browser-based IF interpreter; but it was the first that was completely cross-browser and smooth enough to attract casual viewers. Earlier attempts included Gnusto -- a Mozilla plugin which was Parchment's ancestor -- and Matt Russotto's Zplet, a Java applet.)

I said "absolute necessity" and I meant it. So I started cranking away on a pure-Javascript Glulx interpreter -- the Glulx parallel to Parchment. Sticking to my incomprehensible naming scheme, I called this "Quixe".

Internally Quixe has the same modularity as other Glulx interpreters. There's a VM engine, and a Glk library (although this is a Javascript library rather than a C library). Plus the shim I mentioned earlier. In fact I found it useful to divide the Glk layer itself into two layers -- one which uses the original Glk API model, and one which uses a simpler, web-app-style API. The latter may someday be hooked up to server-side IF through an AJAX connection. I'm not saying that's a good idea; I'm just saying that if you ever want to do it, the library is there for you.

Truly it is said that the hacker whom the Gods would make mad, they first inspire to add one more layer of abstraction. Just one more. Then it'll be perfect.

I released Quixe in July of 2010.

So after all that, what am I proposing for my Kickstarter open-source projects? If you go back and look at my video, you should have a clearer picture of what all the bullet points mean. Or maybe not. Next post: the gory details.

Posted in Zarfplan | Tagged , , , , , | Leave a comment

The techy post, part 2 of 4

Sorry, we've experienced a bit of post bloat. This series will be four parts, not three. (Just four, though.)

Zarf and Modularity

I wanted to design a new 32-bit VM to carry Inform into the Century of the Fruitbat. With (potentially) four gigabytes of memory, and without a host of trivial Z-machine limitations, we'd have room for some serious interactive fiction design. But I wanted to start on a solid foundation.

I was impressed by the generality of the Z-machine design; so I planned a design that went farther. I would completely separate the display layer -- text layout and text input -- from the virtual machine's "CPU" and memory layer.

That sounds wrongheaded, but it grew directly out of my experience in implementing IF interpreters. By 1998 I had released a Z-machine interpreter for X windows, and then ported it to the Mac, and then released a TADS interpreter for the Mac. Comparing the three, it was clear that each application could be divided into a portable VM engine (Z-machine or TADS), and a display library (Mac or X). It really should be possible to create a new VM, and combine it with all the existing display libraries to get instant ports -- or write a new display library, and combine it with all the existing engines.

Moreover, it should be possible to improve the display layer and the VM layer separately. We wouldn't be locked into upgrading the entire thing as a unit. Infocom had done that, near the end of their tenure, producing the "version-6 Z-machine" -- which abandoned much of the portability in order to get a graphical UI out the door. Infocom got a couple of good games that way (Zork Zero, Journey) but they are much harder to play in the modern era than their earlier, cleaner games. I wanted to avoid that mistake. So, modularity it was.

To make this dream work, all I needed to do was specify the API -- the docking port between VM and display library. So I wrote this spec up, and I named it "Glk". To get the ball rolling, I wrote Glk display libraries for Mac (OS8, at the time) and for Linux (both X windows and terminal windows).

(I won't translate "Glk", because it doesn't mean anything.)

I hoped that Glk would change the IF landscape, but it didn't turn out that way. My API did a great job with the basics of text windows and status lines. But IF engines were pushing into the realms of sound, graphics, and mouse input. (Really, that had started in Infocom's day.) Every system handled these things slightly differently, and Glk was slightly different from all of them. Rather than a universal display system, I'd created an adequate common denominator.

Of course, that was a good thing. But the true power of the Glk API wouldn't be realized until I'd created a virtual machine that had no display model of its own -- a VM that relied solely on Glk for input and output.

I worked out that design in 1999; I called it "Glulx". (For translation, see above.) I implemented the VM engine -- in portable C, as promised. Then I turned my attention to the harder problem: adapting the Inform 6 compiler to generate Glulx game files. With that done, people could compile their existing Inform games to Glulx, with only minimal code changes. I didn't expect everybody to jump in and do that with every game -- indeed, I continued to build Z-code games myself. But it provided a smooth transition path.

(I'm glossing over all sorts of details here, you understand. For example, the Glk library doesn't fit perfectly with Glulx game code: one is a C library on the real machine, the other is Inform code on a virtual machine. I needed a shim to translate data structures back and forth. There was also a whole pile of work in translating the Inform 6 library -- the standard parser and so on -- which had a lot of fixed assumptions about the Z-machine's architecture. But this all got done.)

It's worth noting that while I am proud of Glulx, it is not the end-all of IF virtual machines. It was constrained in some ways by the design assumptions of Inform 6 (which was constrained by the design assumptions of the Z-machine). I don't mean that Glulx had to be backwards-compatible with Z-code; it's not. But I6 semantics revolve around pointers, byte arrays, and byte structures (very much like C semantics). So Glulx offers a flat space of bytes for storage. In contrast, modern languages (Perl, Python, Java, Javascript, etc) live in a world of run-time-typed, dynamic, garbage-collected objects -- and Glulx has only the most superficial support for that sort of thing.

This makes life difficult for Inform, which is evolving more modern features with every release. But that's for the next post.

Posted in Zarfplan | Tagged , , , , , | Leave a comment

The techy post, part 1 of 3

I spin a bunch of verbiage on the Kickstarter page (and in the video) about Glulx and Quixe and other silly words. Long-time IF aficionados (with your bottles of aficiolemonade, yes, I know I've made that joke before)... sorry.

Long-time IF aficionados already know what I mean by all those terms; you can tune out now. Contrariwise, if you have no interest in the software architecture of IF interpreters... you can also tune out now. But if you think APIs are cool, read on.

History: The Z-machine

In the beginning was the Z-machine. It was 1979 or so, and a bunch of Infocom hackers needed a virtual machine design to easily port their planned text adventures to the home computers of the era.

The Z-machine is a nice design, only slightly compromised by the nagging inconsistencies of real life. It's a simple 16-bit architecture, with a stack and up to 64 kilobytes of memory -- not too different from the real CPUs that were about to conquer the world. The instruction set includes a few operations customized for IF, though. It has single opcodes to print out text, and read in lines of text (and keystrokes when desired). There's also built-in handling for the world model, which is a big linked tree of data structures representing objects. These data structures can have flags and property lists.

It's important to understand, coming in, that this was an extremely general design. The Z-machine has no built-in notions of treasures, weapons, light, darkness, or even players or rooms. There is no text parser in the specification. All of that was left up to the game author, who had to implement them in a program which ran on the Z-machine.

(Of course Infocom developed a large library of code, including text parsing, room handling, and so on. They shared that from game to game. But any part of that code could be adjusted or customized, if needed, for a particular game. The underlying game engine didn't care; it was the same for every game.)

On the flip side, the Z-machine was also general about how text should be displayed. That was left up to the implementation on each home computer. Most people are familiar with the fixed-width, terminal-style display of old IF -- but the games played equally well when the Macintosh arrived, with nice proportional fonts and a scroll bar on the window.

Rolling on. In the post-Infocom era, the Z-machine game files and interpreters were deconstructed by fans. That led to the first open-source interpreters, which allowed Infocom games to be played on newer computers. (With nice proportional fonts!) It also led to Inform, a fan-made compiler which could generate new Z-machine games. (The Inform language didn't resemble what Infocom had used, but it offered the same functionality.)

Inform and the Z-machine made excellent partners. They weren't the only platform for amateur IF in the 1990s, but for several years they were the most advanced tools that were completely free to use and port. (TADS was shareware through 1996 and its source code was not released until, I think, 1998.) I have used the Z-machine for all of my released IF -- not counting my high school Applesoft efforts.

However, even ten years ago, it was clear that the Z-machine would not suffice forever. The 64k memory limit was already stretched in Infocom's time. They had used bit-shifting tricks to reach 128k of space, and then 256k. Inform games, with a more powerful (and less finickily parsimonious) library, had doubled that again. But this extra space could only be used in a few limited ways -- for game text and function code. IF games had a lot of game text and function code, so the tricks helped. But the Z-machine was still fundamentally a 16-bit system, and it could only address 64k of writable game state. Imagine a machine with 512k of ROM, but only 64k of RAM.

It was at that point that I started plotting my dark schemes.

Posted in Zarfplan | Tagged , , , , , | Leave a comment

Zarf Goes Independent: Hadean Lands

Hadean Lands cover art

Two weeks ago I put up a teaser image, which drew a lot of nice comments and anticipation within the IF community. (Thank you!) But one image in one blog post does not a lifestyle make, and today the real fun begins.

Here's the scoop: at the end of this year, I plan to quit my job and start working on interactive fiction full time. Yes, really. The space madness has struck and it's time to do this thing. I can't do this in my spare time any more; I have too much work to finish in the IF world, and it eats at me every time I leave for my day job.

Go to the Kickstarter page to watch the awesome introductory video! (A Gameshelf production, of course.) You can also play the teaser scene of Hadean Lands on my web site.

IF is bigger this year -- probably the biggest it's been since Infocom fell apart. (You all caught that scene on Big Bang Theory, right?) I don't mean that it's challenging Portal for the hearts and thumbs of the gaming world; but people are interested and people are playing. So I think that IF on the iPhone (iPad, iPod Touch) will sell. And I'm willing to touch the mirror and try it.

You'll note I just linked to Kickstarter. Here's the cone under the scoop: I am asking for donations to get this IF project off the ground. Hadean Lands will be a full-length game -- think Dreamhold size -- so it will take some time to write.

But this is not solely a song of "give Zarf money to write an iPhone text adventure". I have other plans. You know how I keep saying things like "Yes, I certainly do have to add CSS to Glk" and "Yes, I certainly do have to add a clock API to Glulx" and "Yes, for the love of Zog, I certainly do have to add graphics to Quixe"? And then I look sad and complain about my lack of free time? That has to stop.

So, starting in January, I will be splitting my time between Hadean Lands and open-source IF infrastructure projects. (And other iPhone projects to be named later.) One major open-source project, of course, will be a Glulx interpreter for iPhone. That way, anybody (who's signed up as an iPhone developer) can put a Glulx game in the App Store -- complete with artwork, help screens, embedded feelies, Gamecenter integration for achievements... All the trimmings. (If at all possible I'll make this usable with Z-code as well. Yes, iPhone Frotz already exists, but that's not a path for single-game publication.)

Want to support this plan? Give me buckets of money.

Don't have an Apple mobile device? Naturally, you'll want to support my Glulx work regardless. And I would like to do Android (Kindle, anything, etc) versions of this game somday -- those ports just aren't part of the initial schedule.

But I have a special offer: only as a Kickstarter reward, you can pre-order the Special Limited Edition of Hadean Lands, which comes on CD. That will be playable on any Glulx interpreter (Mac, Windows, Linux, or any web browser). This will only be available by donating on Kickstarter during the month of November! So eat up.

And, as they say, thank you for your support.

Posted in Zarfplan | Tagged , , , , , , , | 5 Comments