From 9cdc9a280ba0a0d3ac04d64e5569d626acde7065 Mon Sep 17 00:00:00 2001 From: Benedek Racz Date: Tue, 7 Apr 2020 12:04:06 +0200 Subject: [PATCH] [DOC] Add all API documentation --- README.md | 24 ++++----- doc/source/api/console_reader.rst | 62 +++++++++++++++++++++++ doc/source/api/host.rst | 48 ++++++++++++++++++ doc/source/api/index.rst | 83 ++++++++++++++++++++++++++++++- doc/source/api/spawn_classes.rst | 23 --------- doc/source/api/wexpect_util.rst | 28 +++++++++++ doc/source/conf.py | 6 ++- doc/source/history.rst | 42 +++++++++++++++- doc/source/index.rst | 11 ++-- wexpect/console_reader.py | 3 +- wexpect/host.py | 52 +++++-------------- 11 files changed, 293 insertions(+), 89 deletions(-) create mode 100644 doc/source/api/console_reader.rst create mode 100644 doc/source/api/host.rst delete mode 100644 doc/source/api/spawn_classes.rst create mode 100644 doc/source/api/wexpect_util.rst diff --git a/README.md b/README.md index f519b21..ce838fc 100644 --- a/README.md +++ b/README.md @@ -24,7 +24,7 @@ them automatically. To interract with a child process use `spawn` method: ```python -import wexpect +import wexpect child = wexpect.spawn('cmd.exe') child.expect('>') child.sendline('ls') @@ -49,11 +49,11 @@ structured spawn class. But what is the difference between the old and new and what was the problem with the old? Generally, wexpect (both old and new) has three processes: - - - *host* is our original pyton script/program, which want to launch the child. + + - *host* is our original python script/program, which want to launch the child. - *console* is a process which started by the host, and launches the child. (This is a python script) - *child* is the process which want to be launced. - + The child and the console has a common Windows console, distict from the host. The `legacy_wexpect`'s console is a thin script, almost do nothing. It initializes the Windows's @@ -81,7 +81,7 @@ scripts for duplicating software package installations on different servers. It can be used for automated software testing. Wexpect is in the spirit of Don Libes' Expect, but Wexpect is pure Python. Other Expect-like modules for Python require TCL and Expect or require C extensions to be compiled. Wexpect does not -use C, Expect, or TCL extensions. +use C, Expect, or TCL extensions. Original Pexpect should work on any platform that supports the standard Python pty module. While Wexpect works on Windows platforms. The Wexpect interface focuses on ease of use so that simple @@ -117,13 +117,13 @@ See `after_test` section in [appveyor.yml](appveyor.yml) for more details. The wexpect uses [pbr](https://docs.openstack.org/pbr/latest/) for managing releasing procedures. The versioning is handled by the pbr. The *"master-version"* is the git tag. Pbr derives the package version from the git tags. - + ## Basic behaviour Let's go through the example code: ```python -import wexpect +import wexpect child = wexpect.spawn('cmd.exe') child.expect('>') child.sendline('ls') @@ -139,13 +139,13 @@ child.sendline('exit') Call trace: - ::spawn - - spawn_windows::__init__() + - spawn_windows::__init__() - spawn_windows::_spawn() - Wtty::spawn() - Wtty::startChild() - win32process.CreateProcess() - - + + ### expect() `child.expect('>')` @@ -153,14 +153,14 @@ Call trace: Call trace: - spawn_windows::expect() - - spawn_windows::expect_list() + - spawn_windows::expect_list() - spawn_windows::expect_loop() - spawn_windows::read_nonblocking() - Wtty::read_nonblocking() - Wtty::readConsoleToCursor() - Wtty::readConsole() - __consout.ReadConsoleOutputCharacter() - + ### sendline() diff --git a/doc/source/api/console_reader.rst b/doc/source/api/console_reader.rst new file mode 100644 index 0000000..29549fa --- /dev/null +++ b/doc/source/api/console_reader.rst @@ -0,0 +1,62 @@ +Console reader +============== + +.. automodule:: wexpect.console_reader + + +ConsoleReaderPipe +----------------- + +.. autoclass:: ConsoleReaderPipe + + .. automethod:: __init__ + .. automethod:: read_loop + .. automethod:: suspend_child + .. automethod:: resume_child + .. automethod:: refresh_console + .. automethod:: terminate_child + .. automethod:: isalive + .. automethod:: write + .. automethod:: createKeyEvent + .. automethod:: initConsole + .. automethod:: parseData + .. automethod:: getConsoleOut + .. automethod:: getCoord + .. automethod:: getOffset + .. automethod:: readConsole + .. automethod:: readConsoleToCursor + .. automethod:: interact + .. automethod:: sendeof + .. automethod:: create_connection + .. automethod:: close_connection + .. automethod:: send_to_host + .. automethod:: get_from_host + + +ConsoleReaderSocket +------------------- + +.. autoclass:: ConsoleReaderSocket + + .. automethod:: __init__ + .. automethod:: read_loop + .. automethod:: suspend_child + .. automethod:: resume_child + .. automethod:: refresh_console + .. automethod:: terminate_child + .. automethod:: isalive + .. automethod:: write + .. automethod:: createKeyEvent + .. automethod:: initConsole + .. automethod:: parseData + .. automethod:: getConsoleOut + .. automethod:: getCoord + .. automethod:: getOffset + .. automethod:: readConsole + .. automethod:: readConsoleToCursor + .. automethod:: interact + .. automethod:: sendeof + .. automethod:: create_connection + .. automethod:: close_connection + .. automethod:: send_to_host + .. automethod:: get_from_host diff --git a/doc/source/api/host.rst b/doc/source/api/host.rst new file mode 100644 index 0000000..be4f756 --- /dev/null +++ b/doc/source/api/host.rst @@ -0,0 +1,48 @@ +Host +============= + +.. automodule:: wexpect.host + +Functions +--------- + +.. automethod:: wexpect.host.run + +SpawnPipe +--------- + +.. autoclass:: SpawnPipe + + .. automethod:: __init__ + .. automethod:: expect + .. automethod:: expect_exact + .. automethod:: expect_list + .. automethod:: compile_pattern_list + .. automethod:: send + .. automethod:: sendline + .. automethod:: write + .. automethod:: writelines + .. automethod:: sendeof + .. automethod:: read + .. automethod:: readline + .. automethod:: read_nonblocking + + +SpawnSocket +----------- + +.. autoclass:: SpawnSocket + + .. automethod:: __init__ + .. automethod:: expect + .. automethod:: expect_exact + .. automethod:: expect_list + .. automethod:: compile_pattern_list + .. automethod:: send + .. automethod:: sendline + .. automethod:: write + .. automethod:: writelines + .. automethod:: sendeof + .. automethod:: read + .. automethod:: readline + .. automethod:: read_nonblocking diff --git a/doc/source/api/index.rst b/doc/source/api/index.rst index 16c0265..2dfbbec 100644 --- a/doc/source/api/index.rst +++ b/doc/source/api/index.rst @@ -1,7 +1,88 @@ API documentation ================= +Wexpect symbols +--------------- + +Wexpect package has the following symbols. (Exported by :code:`__all__` in code:`__init__.py`) + +.. _wexpect.spawn: + +**spawn** + +This is the main class interface for Wexpect. Use this class to start and control child applications. +There are two implementation: :class:`wexpect.host.SpawnPipe` uses Windows-Pipe for communicate child. +:class:`wexpect.SpawnSocket` uses TCP socket. Choose the default implementation with +:code:`WEXPECT_SPAWN_CLASS` environment variable, or the :class:`wexpect.host.SpawnPipe` will be +chosen by default. + +.. _wexpect.SpawnPipe: + +**SpawnPipe** + +:class:`wexpect.host.SpawnPipe` is the default spawn class, but you can access it directly with its +exact name. + +.. _wexpect.SpawnSocket: + +**SpawnSocket** + +:class:`wexpect.host.SpawnSocket` is the secondary spawn class, you can access it directly with its +exact name or by setting the :code:`WEXPECT_SPAWN_CLASS` environment variable to :code:`SpawnSocket` + +.. _wexpect.run: + +**run** + +:meth:`wexpect.host.run` runs the given command; waits for it to finish; then returns all output as a string. +This function is similar to :code:`os.system()`. + +.. _wexpect.EOF: + +**EOF** + +:class:`wexpect.wexpect_util.EOF` is an exception. This usually means the child has exited. + +.. _wexpect.TIMEOUT: + +**TIMEOUT** + +:class:`wexpect.wexpect_util.TIMEOUT` raised when a read time exceeds the timeout. + +.. _wexpect.__version__: + +**__version__** + +This gives back the version of the wexpect release. Versioning is handled by the +`pbr `_ package, which derives it from Git tags. + +.. _wexpect.spawn_class_name: + +**spawn_class_name** + +Contains the default spawn class' name even if the user has not specified it. The value can be +:code:`SpawnPipe` or :code:`SpawnSocket` + +.. _wexpect.ConsoleReaderSocket: + +**ConsoleReaderSocket** + +For advanced users only! +:class:`wexpect.console_reader.ConsoleReaderSocket` + +.. _wexpect.ConsoleReaderPipe: + +**ConsoleReaderPipe** + +For advanced users only! +:class:`wexpect.console_reader.ConsoleReaderPipe` + +Wexpect modules +--------------- + .. toctree:: :maxdepth: 2 - spawn_classes + host + wexpect_util + console_reader diff --git a/doc/source/api/spawn_classes.rst b/doc/source/api/spawn_classes.rst deleted file mode 100644 index f05cb24..0000000 --- a/doc/source/api/spawn_classes.rst +++ /dev/null @@ -1,23 +0,0 @@ -Spawn classes -============= - -.. automodule:: wexpect.host - -SpawnPipe ---------- - -.. autoclass:: SpawnPipe - - .. automethod:: __init__ - .. automethod:: expect - .. automethod:: expect_exact - .. automethod:: expect_list - .. automethod:: compile_pattern_list - .. automethod:: send - .. automethod:: sendline - .. automethod:: write - .. automethod:: writelines - .. automethod:: sendeof - .. automethod:: read - .. automethod:: readline - .. automethod:: read_nonblocking diff --git a/doc/source/api/wexpect_util.rst b/doc/source/api/wexpect_util.rst new file mode 100644 index 0000000..7b65df0 --- /dev/null +++ b/doc/source/api/wexpect_util.rst @@ -0,0 +1,28 @@ +Wexpect util +============ + +.. automodule:: wexpect.wexpect_util + +Functions +--------- + +.. automethod:: wexpect.wexpect_util.str2bool +.. automethod:: wexpect.wexpect_util.spam +.. automethod:: wexpect.wexpect_util.init_logger +.. automethod:: wexpect.wexpect_util.split_command_line +.. automethod:: wexpect.wexpect_util.join_args + +ExceptionPexpect +---------------- + +.. autoclass:: ExceptionPexpect + +EOF +--- + +.. autoclass:: EOF + +TIMEOUT +------- + +.. autoclass:: TIMEOUT diff --git a/doc/source/conf.py b/doc/source/conf.py index 5e4ba69..64d35ed 100644 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -54,7 +54,9 @@ release = version # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = [ 'sphinx.ext.autodoc' +extensions = [ + 'sphinx.ext.autodoc', + 'sphinx.ext.intersphinx' ] # Add any paths that contain templates here, relative to this directory. @@ -71,7 +73,7 @@ exclude_patterns = [] # The theme to use for HTML and HTML Help pages. See the documentation for # a list of builtin themes. # -html_theme = 'alabaster' +html_theme = 'default' # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, diff --git a/doc/source/history.rst b/doc/source/history.rst index 96ab393..4eb7b0f 100644 --- a/doc/source/history.rst +++ b/doc/source/history.rst @@ -6,4 +6,44 @@ Wexpect was a one-file code developed at University of Washington. There were se `reference `_ to this code with very few (almost none) documentation nor integration. -This project fixes these limitations, with example codes and pypi integration. +This project fixes these limitations, with example codes, tests, and pypi integration. + +Refactor +^^^^^^^^ + +The original wexpect was a monolite, one-file code, with several structural weaknesses. This leads +me to rewrite the whole code. The first variant of the new structure is delivered with +`v3.2.0 `_. + +.. Note:: + The default is the old variant (:code:`legacy_wexpect`), to use the new you need to set the + :code:`WEXPECT_SPAWN_CLASS` environment variable to :code:`SpawnPipe` or :code:`SpawnSocket`, which + are the two new structured spawn classes. + +Old vs new +^^^^^^^^^^ + +But what is the difference between the old and new and what was the problem with the old? + +Generally, wexpect (both old and new) has three processes: + + - *host* is our original python script/program, which want to launch the child. + - *console* is a process which started by the host, and launches the child. (This is a python script) + - *child* is the process which want to be launced. + +The child and the console has a common Windows console, distict from the host. + +The :code:`legacy_wexpect`'s console is a thin script, almost do nothing. It initializes the Windows's +console, and monitors the host and child processes. The magic is done by the host process, which has +the :code:`switchTo()` and :code:`switchBack()` functions, which (de-) attaches the *child-console* +Windows-console. The host manipulates the child's console directly. This direct manipulation is the +main structural weakness. The following task/use-cases are hard/impossible: + + - thread-safe multiprocessing of the host. + - logging (both console and host) + - using in graphical IDE or with pytest + - This variant is highly depends on the pywin32 package. + +The new structure's console is a thick script. The console process do the major console manipulation, +which is controlled by the host via socket (see SpawnSocket) or named-pipe (SpawnPipe). The host +only process the except-loops. diff --git a/doc/source/index.rst b/doc/source/index.rst index 05c0a29..2bd3c0c 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -5,11 +5,6 @@ Wexpect version |version| :target: https://ci.appveyor.com/project/raczben/wexpect :align: right :alt: Build status - -.. warning:: - **UNDER CONSTRUCTION!!!** - Documentation is in a very preliminary state. I'm learning sphinx and - readthedocs. *Wexpect* is a Windows variant of `Pexpect `_ Wexpect and Pexpect makes Python a better tool for controlling other applications. @@ -41,7 +36,7 @@ To interract with a child process use :code:`spawn` method: .. code-block:: python - import wexpect + import wexpect child = wexpect.spawn('cmd.exe') child.expect('>') child.sendline('ls') @@ -50,11 +45,11 @@ To interract with a child process use :code:`spawn` method: child.sendline('exit') -For more information see [examples](./examples) folder. +For more information see `examples `_ folder. Contents: - + .. toctree:: :maxdepth: 2 diff --git a/wexpect/console_reader.py b/wexpect/console_reader.py index 72dd4e0..5dea8f8 100644 --- a/wexpect/console_reader.py +++ b/wexpect/console_reader.py @@ -4,8 +4,7 @@ Wexpect is a Python module for spawning child applications and controlling them automatically. console_reader Implements a virtual terminal, and starts the child program. -The main wexpect.Spawn class connect to this class to reach the child's terminal. - +The main wexpect.spawn class connect to this class to reach the child's terminal. """ import time diff --git a/wexpect/host.py b/wexpect/host.py index 8a62f85..b9865f3 100644 --- a/wexpect/host.py +++ b/wexpect/host.py @@ -1,38 +1,6 @@ -"""Wexpect is a Windows variant of pexpect https://pexpect.readthedocs.io. - -Wexpect is a Python module for spawning child applications and controlling -them automatically. Wexpect can be used for automating interactive applications -such as ssh, ftp, passwd, telnet, etc. It can be used to a automate setup -scripts for duplicating software package installations on different servers. It -can be used for automated software testing. Wexpect is in the spirit of Don -Libes' Expect, but Wexpect is pure Python. Other Expect-like modules for Python -require TCL and Expect or require C extensions to be compiled. Wexpect does not -use C, Expect, or TCL extensions. - -There are two main interfaces to Wexpect -- the function, run() and the class, -spawn. You can call the run() function to execute a command and return the -output. This is a handy replacement for os.system(). - -For example:: - - wexpect.run('ls -la') - -The more powerful interface is the spawn class. You can use this to spawn an -external child command and then interact with the child by sending lines and -expecting responses. - -For example:: - - child = wexpect.spawn('scp foo myname@host.example.com:.') - child.expect('Password:') - child.sendline(mypassword) - -This works even for commands that ask for passwords or other input outside of -the normal stdio streams. - -Spawn file is the main (aka. host) class of the wexpect. The user call Spawn, which -start the console_reader as a subprocess, which starts the read child. - +"""Host module contains calsses and functions for the host application. These will spawn the child +application. These host classes (and some util classes) are the interface for the user. Handle other +modules as protected. """ import time @@ -88,8 +56,9 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None, The previous code can be replace with the following:: - Examples - ======== + run('scp foo user@example.com:.', events={'(?i)password': mypassword}) + + **Examples** Start the apache daemon on the local machine:: @@ -103,9 +72,6 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None, (command_output, exitstatus) = run ('ls -l /bin', withexitstatus=1) - Tricky Examples - =============== - The following will run SSH and execute 'ls -l' on the remote machine. The password 'secret' will be sent if the '(?i)password' pattern is ever seen:: @@ -114,6 +80,12 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None, This will start mencoder to rip a video from DVD. This will also display progress ticks every 5 seconds as it runs. For example:: + from wexpect import * + def print_ticks(d): + print d['event_count'], + run("mencoder dvd://1 -o video.avi -oac copy -ovc copy", + events={TIMEOUT:print_ticks}, timeout=5) + The 'events' argument should be a dictionary of patterns and responses. Whenever one of the patterns is seen in the command out run() will send the associated response string. Note that you should put newlines in your