### Navigation

- index
- modules |
- next |
- previous |
- Gate One Documentation »
- Developer Documentation »

# `terminal.py` - A Pure Python Terminal Emulator

## About This Module

This crux of this module is the Terminal class which is a pure-Python implementation of the quintessential Unix terminal emulator. It does its best to emulate an xterm and along with that comes support for the majority of the relevant portions of ECMA-48. This includes support for emulating varous VT-* terminal types as well as the "linux" terminal type.

The Terminal class's VT-* emulation support is not complete but it should suffice for most terminal emulation needs (e.g. all your typical command line programs should work wonderfully). If something doesn't look quite right or you need support for certain modes added please feel free to open a ticket on Gate One's issue tracker: https://github.com/liftoff/GateOne/issues

Note that Terminal was written from scratch in order to be as fast as possible. It is extensively commented and implements some interesting patterns in order to maximize execution speed (most notably for things that loop). Some bits of code may seem "un-Pythonic" and/or difficult to grok but understand that this is probably due to optimizations. If you know "a better way" please feel free to submit a patch, open a ticket, or send us an email. There's a reason why open source software is a superior development model!

### Supported Emulation Types

Without any special mode settings or parameters Terminal should effectively emulate the following terminal types:

> - xterm (the most important one)
> - ECMA-48/ANSI X3.64
> - Nearly all the VT-* types: VT-52, VT-100, VT-220, VT-320, VT-420, and VT-520
> - Linux console ("linux")

If you want Terminal to support something else or it's missing a feature from any given terminal type please let us know. We'll implement it!

### What Terminal Doesn't Do

The Terminal class is meant to emulate the display portion of a given terminal. It does not translate keystrokes into escape sequences or special control codes--you'll have to take care of that in your application (or at the client-side like Gate One). It does, however, keep track of many keystroke-specific modes of operation such as Application Cursor Keys and the G0 and G1 charset modes with callbacks that can be used to notify your application when such things change.

### Special Considerations

Many methods inside Terminal start with an underscore. This was done to indicate that such methods shouldn't be called directly (from a program that imported the module). If it was thought that a situation might arise where a method could be used externally by a controlling program, the underscore was omitted.

### Asynchronous Use

To support asynchronous usage (and make everything faster), Terminal was written to support extensive callbacks that are called when certain events are encountered. Here are the events and their callbacks:

Callback Constant (ID) | Called when...
--- | ---
`terminal.CALLBACK_SCROLL_UP` | The terminal is scrolled up (back).
`terminal.CALLBACK_CHANGED` | The screen is changed/updated.
`terminal.CALLBACK_CURSOR_POS` | The cursor position changes.
`terminal.CALLBACK_DSR` | A Device Status Report (DSR) is requested (via the DSR escape sequence).
`terminal.CALLBACK_TITLE` | The terminal title changes (xterm-style)
`terminal.CALLBACK_BELL` | The bell character (^G) is encountered.
`terminal.CALLBACK_OPT` | The special optional escape sequence is encountered.
`terminal.CALLBACK_MODE` | The terminal mode setting changes (e.g. use alternate screen buffer).

Note that CALLBACK_DSR is special in that it in most cases it will be called with arguments. See the code for examples of how and when this happens.

Also, in most cases it is unwise to override CALLBACK_MODE since this method is primarily meant for internal use within the Terminal class.

### Using Terminal

Gate One makes extensive use of the Terminal class and its callbacks. So that's a great place to look for specific examples (gateone.py and termio.py, specifically). Having said that, implementing Terminal is pretty straightforward:

    >>> import terminal
    >>> term = terminal.Terminal(24, 80)
    >>> term.write("This text will be written to the terminal screen.")
    >>> term.dump()
    [u'This text will be written to the terminal screen.                               ',
    <snip>
    u'                                                                                ']

Here's an example with some basic callbacks:

    >>> def mycallback():
    ...     "This will be called whenever the screen changes."
    ...     print("Screen update! Perfect time to dump the terminal screen.")
    ...     print(term.dump()[0]) # Only need to see the top line for this demo =)
    ...     print("Just dumped the screen.")
    >>> import terminal
    >>> term = terminal.Terminal(24, 80)
    >>> term.callbacks[term.CALLBACK_CHANGED] = mycallback
    >>> term.write("This should result in mycallback() being called")
    Screen update! Perfect time to dump the terminal screen.
    This should result in mycallback() being called
    Just dumped the screen.

Note

In testing Gate One it was determined that it is faster to perform the conversion of a terminal screen to HTML on the server side than it is on the client side (via JavaScript anyway).

### About The Scrollback Bufffer

The Terminal class implements a scrollback buffer. Here's how it works: Whenever a `Terminal.scroll_up()` event occurs, the line (or lines) that will be removed from the top of the screen will be placed into `Terminal.scrollback_buf`. Then whenever `Terminal.dump_html()` is called the scrollback buffer will be returned along with the screen output and reset to an empty state.

Why do this? In the event that a very large `Terminal.write()` occurs (e.g. 'ps aux'), it gives the controlling program the ability to capture what went past the screen without some fancy tracking logic surrounding `Terminal.write()`.

More information about how this works can be had by looking at the `Terminal.dump_html()` function itself.

Note

There's more than one function that empties `Terminal.scrollback_buf` when called. You'll just have to have a look around =)

## Class Docstrings

`terminal.``handle_special`(e)

  Used in conjunction with codecs.register_error, will replace special ascii characters such as 0xDA and 0xc4 (which are used by ncurses) with their Unicode equivalents.

`terminal.``_reduce_renditions`(renditions)

  Takes a list, renditions, and reduces it to its logical equivalent (as far as renditions go). Example:

      [0, 32, 0, 34, 0, 32]

  Would become:

      [0, 32]

  Other Examples:

      [0, 1, 36, 36]      ->  [0, 1, 36]
      [0, 30, 42, 30, 42] ->  [0, 30, 42]
      [36, 32, 44, 42]    ->  [32, 42]
      [36, 35]            ->  [35]

`terminal.``pua_counter`()

  A generator that returns a Unicode Private Use Area (PUA) character starting at the beginning of Plane 16 (U+100000); counting up by one with each successive call. If this is a narrow Python build the tail end of Plane 15 will be used as a fallback (with a lot less characters).

  Note

  Meant to be used as references to image objects in the screen array()

class `terminal.``Terminal`(rows=24, cols=80, em_dimensions=None)

  Terminal controller class.

  Initializes the terminal by calling self.initialize(rows, cols). This is so we can have an equivalent function in situations where __init__() gets overridden.

  If em_dimensions are provided they will be used to determine how many lines images will take when they're drawn in the terminal. This is to prevent images that are written to the top of the screen from having their tops cut off. em_dimensions should be a dict in the form of:

      {'height': <px>, 'width': <px>}

  `initialize`(rows=24, cols=80, em_dimensions=None)

    Initializes the terminal (the actual equivalent to `__init__()`).

  `init_screen`()

    Fills `screen` with empty lines of (unicode) spaces using `self.cols` and `self.rows` for the dimensions.

    Note

    Just because each line starts out with a uniform length does not mean it will stay that way. Processing of escape sequences is handled when an output function is called.

  `init_renditions`(rendition=u'U00100000')

    Replaces `self.renditions` with arrays of rendition (characters) using `self.cols` and `self.rows` for the dimenions.

  `init_scrollback`()

    Empties the scrollback buffers (`self.scrollback_buf` and `self.scrollback_renditions`).

  `add_callback`(event, callback, identifier=None)

    Attaches the given callback to the given event. If given, identifier can be used to reference this callback leter (e.g. when you want to remove it). Otherwise an identifier will be generated automatically. If the given identifier is already attached to a callback at the given event that callback will be replaced with callback.

    > event: | The numeric ID of the event you're attaching callback to. The callback constants should be used as the numerical IDs.
    > callback: | The function you're attaching to the event.
    > identifier: | A string or number to be used as a reference point should you wish to remove or update this callback later.

    Returns the identifier of the callback. to Example:

        >>> term = Terminal()
        >>> def somefunc(): pass
        >>> id = "myref"
        >>> ref = term.add_callback(term.CALLBACK_BELL, somefunc, id)

    Note

    This allows the controlling program to have multiple callbacks for the same event.

  `remove_callback`(event, identifier)

    Removes the callback referenced by identifier that is attached to the given event. Example:

        >>> term.remove_callback(CALLBACK_BELL, "myref")

  `remove_all_callbacks`(identifier)

    Removes all callbacks associated with identifier.

  `reset`(*args, **kwargs)

    Resets the terminal back to an empty screen with all defaults. Calls `Terminal.callbacks[CALLBACK_RESET]()` when finished.

    Note

    If terminal output has been suspended (e.g. via ctrl-s) this will not un-suspend it (you need to issue ctrl-q to the underlying program to do that).

  `resize`(rows, cols, em_dimensions=None)

    Resizes the terminal window, adding or removing rows or cols as needed. If em_dimensions are provided they will be stored in self.em_dimensions (which is currently only used by image output).

  `_set_top_bottom`(settings)

    DECSTBM - Sets `self.top_margin` and `self.bottom_margin` using the provided settings in the form of '<top_margin>;<bottom_margin>'.

    Note

    This also handles restore/set "DEC Private Mode Values".

  `get_cursor_position`()

    Returns the current cursor positition as a tuple:

        (row, col)

  `set_title`(title)

    Sets `self.title` to title and executes `Terminal.callbacks[CALLBACK_TITLE]()`

  `get_title`()

    Returns `self.title`

  `save_cursor_position`(mode=None)

    Saves the cursor position and current rendition settings to `self.saved_cursorX`, `self.saved_cursorY`, and `self.saved_rendition`

    Note

    Also handles the set/restore "Private Mode Settings" sequence.

  `restore_cursor_position`(*args, **kwargs)

    Restores the cursor position and rendition settings from `self.saved_cursorX`, `self.saved_cursorY`, and `self.saved_rendition` (if they're set).

  `_dsr_get_cursor_position`()

    Returns the current cursor positition as a DSR response in the form of:

        '<self.cursorY>;<self.cursorX>R'

    Also executes CALLBACK_DSR with the same output as the first argument. Example:

        self.callbacks[CALLBACK_DSR]('20;123R')

  `_dcs_handler`(string=None)

    Handles Device Control String sequences. Unimplemented. Probablye not appropriate for Gate One. If you believe this to be false please open a ticket in the issue tracker.

  `_set_line_params`(param)

    This function handles the control sequences that set double and single line heights and widths. It also handles the "screen alignment test" ( fill the screen with Es).

    Note

    Double-line height text is currently unimplemented (does anything actually use it?).

  `set_G0_charset`(char)

    Sets the terminal's G0 (default) charset to the type specified by char.

    Here's the possibilities:

        0    DEC Special Character and Line Drawing Set
        A    United Kingdom (UK)
        B    United States (USASCII)
        4    Dutch
        C    Finnish
        5    Finnish
        R    French
        Q    French Canadian
        K    German
        Y    Italian
        E    Norwegian/Danish
        6    Norwegian/Danish
        Z    Spanish
        H    Swedish
        7    Swedish
        =    Swiss

  `set_G1_charset`(char)

    Sets the terminal's G1 (alt) charset to the type specified by char.

    Here's the possibilities:

        0    DEC Special Character and Line Drawing Set
        A    United Kingdom (UK)
        B    United States (USASCII)
        4    Dutch
        C    Finnish
        5    Finnish
        R    French
        Q    French Canadian
        K    German
        Y    Italian
        E    Norwegian/Danish
        6    Norwegian/Danish
        Z    Spanish
        H    Swedish
        7    Swedish
        =    Swiss

  `use_g0_charset`()

    Sets the current charset to G0. This should get called when ASCII_SO is encountered.

  `use_g1_charset`()

    Sets the current charset to G1. This should get called when ASCII_SI is encountered.

  `write`(chars, special_checks=True)

    Write chars to the terminal at the current cursor position advancing the cursor as it does so. If chars is not unicode, it will be converted to unicode before being stored in self.screen.

    if special_checks is True (default), Gate One will perform checks for special things like image files coming in via chars.

  `flush`()

    Only here to make Terminal compatible with programs that want to use file-like methods.

  `scroll_up`(n=1)

    Scrolls up the terminal screen by n lines (default: 1). The callbacks CALLBACK_CHANGED and CALLBACK_SCROLL_UP are called after scrolling the screen.

    Note

    This will only scroll up the region within self.top_margin and self.bottom_margin (if set).

  `scroll_down`(n=1)

    Scrolls down the terminal screen by n lines (default: 1). The callbacks CALLBACK_CHANGED and CALLBACK_SCROLL_DOWN are called after scrolling the screen.

  `insert_line`(n=1)

    Inserts n lines at the current cursor position.

  `delete_line`(n=1)

    Deletes n lines at the current cursor position.

  `backspace`()

    Execute a backspace (x08)

  `horizontal_tab`()

    Execute horizontal tab (x09)

  `_set_tabstop`()

    Sets a tabstop at the current position of `self.cursorX`.

  `linefeed`()

    LF - Executes a line feed.

    Note

    This actually just calls `Terminal.newline()`.

  `next_line`()

    CNL - Moves the cursor down one line to the home position. Will not result in a scrolling event like newline() does.

    Note

    This is not the same thing as `Terminal.cursor_next_line()` which preserves the cursor's column position.

  `reverse_linefeed`()

    RI - Executes a reverse line feed: Move the cursor up one line to the home position. If the cursor move would result in going past the top margin of the screen (upwards) this will execute a scroll_down() event.

  `newline`()

    Increases `self.cursorY` by 1 and calls `Terminal.scroll_up()` if that action will move the curor past `self.bottom_margin` (usually the bottom of the screen).

  `_carriage_return`()

    Executes a carriage return (sets `self.cursorX` to 0). In other words it moves the cursor back to position 0 on the line.

  `_xon`()

    Handles the XON character (stop ignoring).

    Note

    Doesn't actually do anything (this feature was probably meant for the underlying terminal program).

  `_xoff`()

    Handles the XOFF character (start ignoring)

    Note

    Doesn't actually do anything (this feature was probably meant for the underlying terminal program).

  `_cancel_esc_sequence`()

    Cancels any escape sequence currently being processed. In other words it empties `self.esc_buffer`.

  `_sub_esc_sequence`()

    Cancels any escape sequence currently in progress and replaces `self.esc_buffer` with single question mark (?).

    Note

    Nothing presently uses this function and I can't remember what it was supposed to be part of (LOL!). Obviously it isn't very important.

  `_escape`()

    Handles the escape character as well as escape sequences that may end with an escape character.

  `_csi`()

    Marks the start of a CSI escape sequence (which is itself a character) by setting `self.esc_buffer` to '\x1b[' (which is the CSI escape sequence).

  `_capture_image`()

    This gets called when an image (PNG or JPEG) was detected by `Terminal.write()` and captured in `self.image`. It cleans up the data inside `self.image` (getting rid of carriage returns) and stores a reference to self.image as a single character (using `self.image_counter`) in `self.screen` at the current cursor position. The actual image will be written to disk and read on-demand by `self__spanify_screen()` when it needs to be displayed. The image on disk will be automatically removed when it is no longer visible.

    It also moves the cursor to the beginning of the last line before doing this in order to ensure that the captured image stays within the browser's current window.

    Note

    The `Terminal._spanify_screen()` function is aware of this logic and knows that a 'character' in Plane 16 of the Unicode PUA indicates the presence of something like an image that needs special processing.

  `_image_fd_watcher`()

    Meant to be run inside of a thread, calls close_image_fds() until there are no more open image file descriptors.

  `close_image_fds`()

    Closes the file descriptors of any images that are no longer on the screen.

  `_string_terminator`()

    Handle the string terminator (ST).

    Note

    Doesn't actually do anything at the moment. Probably not needed since `Terminal._escape()` and/or `Terminal.bell()` will end up handling any sort of sequence that would end in an ST anyway.

  `_osc_handler`()

    Handles Operating System Command (OSC) escape sequences which need special care since they are of indeterminiate length and end with either a bell (\x07) or a sequence terminator (\x9c aka ST). This will usually be called from `Terminal.bell()` to set the title of the terminal (just like an xterm) but it is also possible to be called directly whenever an ST is encountered.

  `bell`()

    Handles the bell character and executes `Terminal.callbacks[CALLBACK_BELL]()` (if we are not in the middle of an escape sequence that ends with a bell character =). If we are in the middle of an escape sequence, calls `self._osc_handler()` since we can be nearly certain that we're simply terminating an OSC sequence. Isn't terminal emulation grand? ⨀_⨀

  `_device_status_report`()

    Returns '[0n' (terminal OK) and executes:

        self.callbacks[CALLBACK_DSR]("\x1b[0n")

  `_csi_device_status_report`(request)

    Returns '\x1b[1;2c' (Meaning: I'm a vt220 terminal, version 1.0) and executes:

        self.callbacks[self.CALLBACK_DSR]("\x1b[1;2c")

  `set_expanded_mode`(setting)

    Accepts "standard mode" settings. Typically '\x1b[?25h' to hide cursor.

    Notes on modes:

        '?1h' - Application Cursor Keys
        '?5h' - DECSCNM (default off): Set reverse-video mode.
        '?12h' - Local echo (SRM or Send Receive Mode)
        '?25h' - Hide cursor
        '?1049h' - Save cursor and screen

  `reset_expanded_mode`(setting)

    Accepts "standard mode" settings. Typically '\x1b[?25l' to show cursor.

  `set_application_mode`(boolean)

    Sets `self.application_keys` equal to boolean. Literally:

        self.application_keys = boolean

  `toggle_alternate_screen_buffer`(alt)

    If alt is True, copy the current screen and renditions to `self.alt_screen` and `self.alt_renditions` then re-init `self.screen` and `self.renditions`.

    If alt is False, restore the saved screen buffer and renditions then nullify `self.alt_screen` and `self.alt_renditions`.

  `toggle_alternate_screen_buffer_cursor`(alt)

    Same as `Terminal.toggle_alternate_screen_buffer()` but also saves/restores the cursor location.

  `show_hide_cursor`(boolean)

    Literally:

        self.show_cursor = boolean

  `send_receive_mode`(onoff)

    Turns on or off local echo dependong on the value of onoff:

        self.local_echo = onoff

  `insert_characters`(n=1)

    Inserts the specified number of characters at the cursor position.

  `delete_characters`(n=1)

    DCH - Deletes (to the left) the specified number of characters at the cursor position. As characters are deleted, the remaining characters between the cursor and right margin move to the left. Character attributes (renditions) move with the characters. The terminal adds blank spaces with no visual character attributes at the right margin. DCH has no effect outside the scrolling margins.

    Note

    Deletes renditions too. You'd think that would be in one of the VT-* manuals... Nope!

  `_erase_characters`(n=1)

    Erases (to the right) the specified number of characters at the cursor position.

    Note

    Deletes renditions too.

  `cursor_left`(n=1)

    ESCnD CUB (Cursor Back)

  `cursor_right`(n=1)

    ESCnC CUF (Cursor Forward)

  `cursor_up`(n=1)

    ESCnA CUU (Cursor Up)

  `cursor_down`(n=1)

    ESCnB CUD (Cursor Down)

  `cursor_next_line`(n)

    ESCnE CNL (Cursor Next Line)

  `cursor_previous_line`(n)

    ESCnF CPL (Cursor Previous Line)

  `cursor_horizontal_absolute`(n)

    ESCnG CHA (Cursor Horizontal Absolute)

  `cursor_position`(coordinates)

    ESCnH CUP (Cursor Position). Move the cursor to the given coordinates.

    > coordinates: | Should be something like, 'row;col' (1-based) but, 'row', 'row;', and ';col' are also valid (assumes 1 on missing value).

    Note

    If coordinates is '' (an empty string), the cursor will be moved to the top left (1;1).

  `cursor_position_vertical`(n)

    Vertical Line Position Absolute (VPA) - Moves the cursor to given line.

  `clear_screen`()

    Clears the screen. Also used to emulate a terminal reset.

    Note

    The current rendition (self.cur_rendition) will be applied to all characters on the screen when this function is called.

  `clear_screen_from_cursor_down`()

    Clears the screen from the cursor down (ESC[J or ESC[0J).

  `clear_screen_from_cursor_up`()

    Clears the screen from the cursor up (ESC[1J).

  `clear_screen_from_cursor`(n)

    CSI n J ED (Erase Data). This escape sequence uses the following rules:

    Esc[J | Clear screen from cursor down | ED0
    Esc[0J | Clear screen from cursor down | ED0
    Esc[1J | Clear screen from cursor up | ED1
    Esc[2J | Clear entire screen | ED2

  `clear_line_from_cursor_right`()

    Clears the screen from the cursor right (ESC[K or ESC[0K).

  `clear_line_from_cursor_left`()

    Clears the screen from the cursor left (ESC[1K).

  `clear_line`()

    Clears the entire line (ESC[2K).

  `clear_line_from_cursor`(n)

    CSI*n*K EL (Erase in Line). This escape sequence uses the following rules:

    Esc[K | Clear screen from cursor right | EL0
    Esc[0K | Clear screen from cursor right | EL0
    Esc[1K | Clear screen from cursor left | EL1
    Esc[2K | Clear entire line | ED2

  `set_led_state`(n)

    Sets the values the dict, self.leds depending on n using the following rules:

    Esc[0q | Turn off all four leds | DECLL0
    Esc[1q | Turn on LED #1 | DECLL1
    Esc[2q | Turn on LED #2 | DECLL2
    Esc[3q | Turn on LED #3 | DECLL3
    Esc[4q | Turn on LED #4 | DECLL4

    Note

    These aren't implemented in Gate One's GUI (yet) but they certainly kept track of!

  `_set_rendition`(n)

    Sets `self.renditions[self.cursorY][self.cursorX]` equal to n.split(';').

    n is expected to be a string of ECMA-48 rendition numbers separated by semicolons. Example:

        '0;1;31'

    ...will result in:

        [0, 1, 31]

    Note that the numbers were converted to integers and the order was preserved.

  `_opt_handler`(chars)

    Optional special escape sequence handler for sequences matching RE_OPT_SEQ. If CALLBACK_OPT is defined it will be called like so:

        self.callbacks[CALLBACK_OPT](chars)

    Applications can use this escape sequence to define whatever special handlers they like. It works like this: If an escape sequence is encountered matching RE_OPT_SEQ this method will be called with the inbetween chars (e.g. ]_;<chars>) as the argument.

    Applications can then do what they wish with chars.

    Note

    I added this functionality so that plugin authors would have a mechanism to communicate with terminal applications. See the SSH plugin for an example of how this can be done (there's channels of communication amongst ssh_connect.py, ssh.js, and ssh.py).

  `_spanify_screen`()

    Iterates over the lines in screen and renditions, applying HTML markup (span tags) where appropriate and returns the result as a list of lines. It also marks the cursor position via a <span> tag at the appropriate location.

  `_spanify_scrollback`()

    Spanifies everything inside screen using renditions. This differs from _spanify_screen() in that it doesn't apply any logic to detect the location of the cursor (to make it just a tiny bit faster).

  `dump_html`(renditions=True)

    Dumps the terminal screen as a list of HTML-formatted lines. If renditions is True (default) then terminal renditions will be converted into HTML <span> elements so they will be displayed properly in a browser. Otherwise only the cursor <span> will be added to mark its location.

    Note

    This places <span class="cursor">(current character)</span> around the cursor location.

  `dump_plain`()

    Dumps the screen and the scrollback buffer as-is then empties the scrollback buffer.

  `dump_components`()

    Dumps the screen and renditions as-is, the scrollback buffer as HTML, and the current cursor coordinates. Also, empties the scrollback buffer

    Note

    This was used in some performance-related experiments but might be useful for other patterns in the future so I've left it here.

  `dump`()

    Returns self.screen as a list of strings with no formatting. No scrollback buffer. No renditions. It is meant to be used to get a quick glance of what is being displayed (when debugging).

    Note

    This method does not empty the scrollback buffer.

  `_Terminal__ignore`(*args, **kwargs)

    Does nothing (on purpose!). Used as a placeholder for unimplemented functions.

`terminal.``css_colors`(selector=None)

  Returns a (long) string containing all the CSS styles in order to support color in an HTML terminal using the dump_html() function. If selector is provided, all styles will be prefixed with said selector like so:

      ${selector} span.f0 { color: #5C5C5C; }

  Example:

      >>> css_colors("#gateone").splitlines()[1]
      '#gateone span.f0 { color: #5C5C5C; } /* Black */'

### Navigation

- index
- modules |
- next |
- previous |
- Gate One Documentation »
- Developer Documentation »