[DOC] Add all API documentation

README.md

@@ -50,7 +50,7 @@ But what is the difference between the old and new and what was the problem with
 
 Generally, wexpect (both old and new) has three processes:
 
- - *host* is our original pyton script/program, which want to launch the child.
+ - *host* is our original python script/program, which want to launch the child.
  - *console* is a process which started by the host, and launches the child. (This is a python script)
  - *child* is the process which want to be launced.
 

doc/source/api/console_reader.rst

@@ -0,0 +1,62 @@
+Console reader
+==============
+
+.. automodule:: wexpect.console_reader
+
+
+ConsoleReaderPipe
+-----------------
+
+.. autoclass:: ConsoleReaderPipe
+
+  .. automethod:: __init__
+  .. automethod:: read_loop
+  .. automethod:: suspend_child
+  .. automethod:: resume_child
+  .. automethod:: refresh_console
+  .. automethod:: terminate_child
+  .. automethod:: isalive
+  .. automethod:: write
+  .. automethod:: createKeyEvent
+  .. automethod:: initConsole
+  .. automethod:: parseData
+  .. automethod:: getConsoleOut
+  .. automethod:: getCoord
+  .. automethod:: getOffset
+  .. automethod:: readConsole
+  .. automethod:: readConsoleToCursor
+  .. automethod:: interact
+  .. automethod:: sendeof
+  .. automethod:: create_connection
+  .. automethod:: close_connection
+  .. automethod:: send_to_host
+  .. automethod:: get_from_host
+
+
+ConsoleReaderSocket
+-------------------
+
+.. autoclass:: ConsoleReaderSocket
+
+  .. automethod:: __init__
+  .. automethod:: read_loop
+  .. automethod:: suspend_child
+  .. automethod:: resume_child
+  .. automethod:: refresh_console
+  .. automethod:: terminate_child
+  .. automethod:: isalive
+  .. automethod:: write
+  .. automethod:: createKeyEvent
+  .. automethod:: initConsole
+  .. automethod:: parseData
+  .. automethod:: getConsoleOut
+  .. automethod:: getCoord
+  .. automethod:: getOffset
+  .. automethod:: readConsole
+  .. automethod:: readConsoleToCursor
+  .. automethod:: interact
+  .. automethod:: sendeof
+  .. automethod:: create_connection
+  .. automethod:: close_connection
+  .. automethod:: send_to_host
+  .. automethod:: get_from_host

doc/source/api/host.rst

@@ -0,0 +1,48 @@
+Host
+=============
+
+.. automodule:: wexpect.host
+
+Functions
+---------
+
+.. automethod:: wexpect.host.run
+
+SpawnPipe
+---------
+
+.. autoclass:: SpawnPipe
+
+  .. automethod:: __init__
+  .. automethod:: expect
+  .. automethod:: expect_exact
+  .. automethod:: expect_list
+  .. automethod:: compile_pattern_list
+  .. automethod:: send
+  .. automethod:: sendline
+  .. automethod:: write
+  .. automethod:: writelines
+  .. automethod:: sendeof
+  .. automethod:: read
+  .. automethod:: readline
+  .. automethod:: read_nonblocking
+
+
+SpawnSocket
+-----------
+
+.. autoclass:: SpawnSocket
+
+  .. automethod::  __init__
+  .. automethod:: expect
+  .. automethod:: expect_exact
+  .. automethod:: expect_list
+  .. automethod:: compile_pattern_list
+  .. automethod:: send
+  .. automethod:: sendline
+  .. automethod:: write
+  .. automethod:: writelines
+  .. automethod:: sendeof
+  .. automethod:: read
+  .. automethod:: readline
+  .. automethod:: read_nonblocking

doc/source/api/index.rst

@@ -1,7 +1,88 @@
 API documentation
 =================
 
+Wexpect symbols
+---------------
+
+Wexpect package has the following symbols. (Exported by :code:`__all__` in code:`__init__.py`)
+
+.. _wexpect.spawn:
+
+**spawn**
+
+This is the main class interface for Wexpect. Use this class to start and control child applications.
+There are two implementation: :class:`wexpect.host.SpawnPipe` uses Windows-Pipe for communicate child.
+:class:`wexpect.SpawnSocket` uses TCP socket. Choose the default implementation with
+:code:`WEXPECT_SPAWN_CLASS` environment variable, or the :class:`wexpect.host.SpawnPipe` will be
+chosen by default.
+
+.. _wexpect.SpawnPipe:
+
+**SpawnPipe**
+
+:class:`wexpect.host.SpawnPipe` is the default spawn class, but you can access it directly with its
+exact name.
+
+.. _wexpect.SpawnSocket:
+
+**SpawnSocket**
+
+:class:`wexpect.host.SpawnSocket` is the secondary spawn class, you can access it directly with its
+exact name or by setting the :code:`WEXPECT_SPAWN_CLASS` environment variable to :code:`SpawnSocket`
+
+.. _wexpect.run:
+
+**run**
+
+:meth:`wexpect.host.run` runs the given command; waits for it to finish; then returns all output as a string.
+This function is similar to :code:`os.system()`.
+
+.. _wexpect.EOF:
+
+**EOF**
+
+:class:`wexpect.wexpect_util.EOF` is an exception. This usually means the child has exited.
+
+.. _wexpect.TIMEOUT:
+
+**TIMEOUT**
+
+:class:`wexpect.wexpect_util.TIMEOUT` raised when a read time exceeds the timeout.
+
+.. _wexpect.__version__:
+
+**__version__**
+
+This gives back the version of the wexpect release. Versioning is handled by the
+`pbr <https://pypi.org/project/pbr/>`_ package, which derives it from Git tags.
+
+.. _wexpect.spawn_class_name:
+
+**spawn_class_name**
+
+Contains the default spawn class' name even if the user has not specified it. The value can be
+:code:`SpawnPipe` or :code:`SpawnSocket`
+
+.. _wexpect.ConsoleReaderSocket:
+
+**ConsoleReaderSocket**
+
+For advanced users only!
+:class:`wexpect.console_reader.ConsoleReaderSocket`
+
+.. _wexpect.ConsoleReaderPipe:
+
+**ConsoleReaderPipe**
+
+For advanced users only!
+:class:`wexpect.console_reader.ConsoleReaderPipe`
+
+Wexpect modules
+---------------
+
 .. toctree::
    :maxdepth: 2
 
-   spawn_classes
+   host
+   wexpect_util
+   console_reader

doc/source/api/spawn_classes.rst

@@ -1,23 +0,0 @@
-Spawn classes
-=============
-
-.. automodule:: wexpect.host
-
-SpawnPipe
----------
-
-.. autoclass:: SpawnPipe
-
-   .. automethod:: __init__
-   .. automethod:: expect
-   .. automethod:: expect_exact
-   .. automethod:: expect_list
-   .. automethod:: compile_pattern_list
-   .. automethod:: send
-   .. automethod:: sendline
-   .. automethod:: write
-   .. automethod:: writelines
-   .. automethod:: sendeof
-   .. automethod:: read
-   .. automethod:: readline
-   .. automethod:: read_nonblocking

doc/source/api/wexpect_util.rst

@@ -0,0 +1,28 @@
+Wexpect util
+============
+
+.. automodule:: wexpect.wexpect_util
+
+Functions
+---------
+
+.. automethod:: wexpect.wexpect_util.str2bool
+.. automethod:: wexpect.wexpect_util.spam
+.. automethod:: wexpect.wexpect_util.init_logger
+.. automethod:: wexpect.wexpect_util.split_command_line
+.. automethod:: wexpect.wexpect_util.join_args
+
+ExceptionPexpect
+----------------
+
+.. autoclass:: ExceptionPexpect
+
+EOF
+---
+
+.. autoclass:: EOF
+
+TIMEOUT
+-------
+
+.. autoclass:: TIMEOUT

doc/source/conf.py

@@ -54,7 +54,9 @@ release = version
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = [ 'sphinx.ext.autodoc'
+extensions = [
+    'sphinx.ext.autodoc',
+    'sphinx.ext.intersphinx'
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -71,7 +73,7 @@ exclude_patterns = []
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'alabaster'
+html_theme = 'default'
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,

doc/source/history.rst

@@ -6,4 +6,44 @@ Wexpect was a one-file code developed at University of Washington. There were se
 `reference <https://mediarealm.com.au/articles/python-pexpect-windows-wexpect/>`_
 to this code with very few (almost none) documentation nor integration.
 
-This project fixes these limitations, with example codes and pypi integration.
+This project fixes these limitations, with example codes, tests, and pypi integration.
+
+Refactor
+^^^^^^^^
+
+The original wexpect was a monolite, one-file code, with several structural weaknesses. This leads
+me to rewrite the whole code. The first variant of the new structure is delivered with
+`v3.2.0 <https://pypi.org/project/wexpect/3.2.0/>`_.
+
+.. Note::
+    The default is the old variant (:code:`legacy_wexpect`), to use the new you need to set the
+    :code:`WEXPECT_SPAWN_CLASS` environment variable to :code:`SpawnPipe` or :code:`SpawnSocket`, which
+    are the two new structured spawn classes.
+
+Old vs new
+^^^^^^^^^^
+
+But what is the difference between the old and new and what was the problem with the old?
+
+Generally, wexpect (both old and new) has three processes:
+
+ - *host* is our original python script/program, which want to launch the child.
+ - *console* is a process which started by the host, and launches the child. (This is a python script)
+ - *child* is the process which want to be launced.
+
+The child and the console has a common Windows console, distict from the host.
+
+The :code:`legacy_wexpect`'s console is a thin script, almost do nothing. It initializes the Windows's
+console, and monitors the host and child processes. The magic is done by the host process, which has
+the :code:`switchTo()` and :code:`switchBack()` functions, which (de-) attaches the *child-console*
+Windows-console. The host manipulates the child's console directly. This direct manipulation is the
+main structural weakness. The following task/use-cases are hard/impossible:
+
+  - thread-safe multiprocessing of the host.
+  - logging (both console and host)
+  - using in graphical IDE or with pytest
+  - This variant is highly depends on the pywin32 package.
+
+The new structure's console is a thick script. The console process do the major console manipulation,
+which is controlled by the host via socket (see SpawnSocket) or named-pipe (SpawnPipe). The host
+only process the except-loops.

doc/source/index.rst

@@ -6,11 +6,6 @@ Wexpect version |version|
    :align: right
    :alt: Build status
 
-.. warning::
-    **UNDER CONSTRUCTION!!!**
-    Documentation is in a very preliminary state. I'm learning sphinx and
-    readthedocs.
-
 *Wexpect* is a Windows variant of `Pexpect <https://pexpect.readthedocs.io/en/stable/>`_
 Wexpect and Pexpect makes Python a better tool for controlling other applications.
 
@@ -50,7 +45,7 @@ To interract with a child process use :code:`spawn` method:
   child.sendline('exit')
 
 
-For more information see [examples](./examples) folder.
+For more information see `examples <https://github.com/raczben/wexpect/tree/master/examples>`_ folder.
 
 
 Contents:

wexpect/console_reader.py

@@ -4,8 +4,7 @@ Wexpect is a Python module for spawning child applications and controlling
 them automatically.
 
 console_reader Implements a virtual terminal, and starts the child program.
-The main wexpect.Spawn class connect to this class to reach the child's terminal.
+The main wexpect.spawn class connect to this class to reach the child's terminal.
-
 """
 
 import time

wexpect/host.py

@@ -1,38 +1,6 @@
-"""Wexpect is a Windows variant of pexpect https://pexpect.readthedocs.io.
+"""Host module contains calsses and functions for the host application. These will spawn the child
-
+application. These host classes (and some util classes) are the interface for the user. Handle other
-Wexpect is a Python module for spawning child applications and controlling
+modules as protected.
-them automatically. Wexpect can be used for automating interactive applications
-such as ssh, ftp, passwd, telnet, etc. It can be used to a automate setup
-scripts for duplicating software package installations on different servers. It
-can be used for automated software testing. Wexpect is in the spirit of Don
-Libes' Expect, but Wexpect is pure Python. Other Expect-like modules for Python
-require TCL and Expect or require C extensions to be compiled. Wexpect does not
-use C, Expect, or TCL extensions.
-
-There are two main interfaces to Wexpect -- the function, run() and the class,
-spawn. You can call the run() function to execute a command and return the
-output. This is a handy replacement for os.system().
-
-For example::
-
-    wexpect.run('ls -la')
-
-The more powerful interface is the spawn class. You can use this to spawn an
-external child command and then interact with the child by sending lines and
-expecting responses.
-
-For example::
-
-    child = wexpect.spawn('scp foo myname@host.example.com:.')
-    child.expect('Password:')
-    child.sendline(mypassword)
-
-This works even for commands that ask for passwords or other input outside of
-the normal stdio streams.
-
-Spawn file is the main (aka. host) class of the wexpect. The user call Spawn, which
-start the console_reader as a subprocess, which starts the read child.
-
 """
 
 import time
@@ -88,8 +56,9 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None,
 
     The previous code can be replace with the following::
 
-    Examples
+        run('scp foo user@example.com:.', events={'(?i)password': mypassword})
-    ========
+
+    **Examples**
 
     Start the apache daemon on the local machine::
 
@@ -103,9 +72,6 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None,
 
         (command_output, exitstatus) = run ('ls -l /bin', withexitstatus=1)
 
-    Tricky Examples
-    ===============
-
     The following will run SSH and execute 'ls -l' on the remote machine. The
     password 'secret' will be sent if the '(?i)password' pattern is ever seen::
 
@@ -114,6 +80,12 @@ def run(command, timeout=-1, withexitstatus=False, events=None, extra_args=None,
     This will start mencoder to rip a video from DVD. This will also display
     progress ticks every 5 seconds as it runs. For example::
 
+        from wexpect import *
+        def print_ticks(d):
+            print d['event_count'],
+        run("mencoder dvd://1 -o video.avi -oac copy -ovc copy",
+            events={TIMEOUT:print_ticks}, timeout=5)
+
     The 'events' argument should be a dictionary of patterns and responses.
     Whenever one of the patterns is seen in the command out run() will send the
     associated response string. Note that you should put newlines in your