Changeset 963


Ignore:
Timestamp:
Jun 22, 2012 5:58:50 PM (11 years ago)
Author:
jemian
Message:

refs #8, big progress today, now able to get ReST-formatted extended comments from SPEC .mac files into Sphinx output using autodoc and subclassing the autodoc.Documenter class for SPEC.

Also, breaking up the test document into parts to make it easier to follow.

Location:
specdomain/src/specdomain
Files:
5 added
1 deleted
6 edited

Legend:

Unmodified
Added
Removed

  • specdomain/src/specdomain/doc/index.rst

    r937 r963  
    1919   :maxdepth: 2
    2020
    21    reference
    2221   changes
    2322   license

  • specdomain/src/specdomain/sphinxcontrib/specdomain.py

    r961 r963  
    2020import re
    2121import string                                           #@UnusedImport
     22import sys
    2223
    2324from docutils import nodes                              #@UnusedImport
    … …  
    3637from docutils.statemachine import ViewList, string2lines
    3738import sphinx.util.nodes
     39from sphinx.ext.autodoc import Documenter, bool_option
     40from sphinx.util.inspect import getargspec, isdescriptor, safe_getmembers, \
     41     safe_getattr, safe_repr
     42from sphinx.util.pycompat import base_exception, class_types
     43from specmacrofileparser import SpecMacrofileParser
    3844
    3945
    … …  
    7379                                                + r'$',
    7480                                                re.IGNORECASE|re.DOTALL|re.MULTILINE)
     81
     82
     83class SpecMacroDocumenter(Documenter):
     84    """
     85    Document a SPEC macro source code file (autodoc.Documenter subclass)
     86   
     87    This code responds to the ReST file directive::
     88   
     89        .. autospecmacro:: partial/path/name/somefile.mac
     90            :displayorder: fileorder
     91   
     92    The ``:displayorder`` parameter indicates how the
     93    contents will be sorted for appearance in the ReST document.
     94   
     95        **fileorder** or **file**
     96            Items will be documented in the order in
     97            which they appear in the ``.mac`` file.
     98       
     99        **alphabetical** or **alpha**
     100            Items will be documented in alphabetical order.
     101   
     102    .. tip::
     103        A (near) future enhancement will provide for
     104        documenting all macro files in a directory, with optional
     105        recursion into subdirectories.  By default, the code will
     106        only document files that match the glob pattern ``*.mac``.
     107        (This could be defined as a list in the ``conf.py`` file.)
     108        Such as::
     109       
     110           .. spec:directory:: partial/path/name
     111              :recursion:
     112              :displayorder: alphabetical
     113    """
     114
     115    objtype = 'specmacro'
     116    member_order = 50
     117    priority = 0
     118
     119    option_spec = {
     120        'displayorder': bool_option,
     121    }
     122
     123    @classmethod
     124    def can_document_member(cls, member, membername, isattr, parent):
     125        # don't document submodules automatically
     126        #return isinstance(member, (FunctionType, BuiltinFunctionType))
     127        r = membername in ('SpecMacroDocumenter', )
     128        return r
     129   
     130    def generate(self, *args, **kw):
     131        """
     132        Generate reST for the object given by *self.name*, and possibly for
     133        its members.
     134
     135        If *more_content* is given, include that content. If *real_modname* is
     136        given, use that module name to find attribute docs. If *check_module* is
     137        True, only generate if the object is defined in the module name it is
     138        imported from. If *all_members* is True, document all members.
     139        """
     140        # now, parse the SPEC macro file
     141        macrofile = self.parse_name()
     142        spec = SpecMacrofileParser(macrofile)
     143        extended_comment = spec.ReST()
     144       
     145        # FIXME:
     146        #     Assume all extended comments contain ReST formatted comments,
     147        #     *including initial section titles or transitions*.
     148        '''
     149            cdef-examples.mac:7: SEVERE: Unexpected section title.
     150           
     151            Examples of SPEC cdef macros
     152            ==============================
     153            test-battery.mac:4: SEVERE: Unexpected section title or transition.
     154           
     155            ###############################################################################
     156            test-battery.mac:6: WARNING: Block quote ends without a blank line; unexpected unindent.
     157            test-battery.mac:6: SEVERE: Unexpected section title or transition.
     158           
     159            ###############################################################################
     160            test-battery.mac:19: SEVERE: Unexpected section title.
     161           
     162            common/shutter
     163            ==============
     164        '''
     165
     166        rest = prepare_docstring(extended_comment)
     167
     168        # TODO: Another step should (like for Python) attach source code and provide
     169        #       links from each to highlighted source code blocks.
     170
     171        #self.add_line(u'', '<autodoc>')
     172        #sig = self.format_signature()
     173        #self.add_directive_header(sig)
     174        self.add_line(u'', '<autodoc>')
     175        for linenumber, line in enumerate(rest):
     176            self.add_line(line, macrofile, linenumber)
     177        #self.add_content(rest)
     178        #self.document_members(all_members)
     179
     180    def resolve_name(self, modname, parents, path, base):
     181        if modname is not None:
     182            self.directive.warn('"::" in autospecmacro name doesn\'t make sense')
     183        return (path or '') + base, []
     184
     185    def parse_name(self):
     186        """Determine what file to parse.
     187       
     188        :returns: True if if parsing was successful
     189
     190        .. Note:: The template method from autodoc sets *self.modname*, *self.objpath*, *self.fullname*,
     191            *self.args* and *self.retann*.  This is not done here yet.
     192        """
     193        ret = self.name
     194        self.fullname = os.path.abspath(ret)        # TODO: Consider using this
     195        self.fullname = ret                         # TODO: provisional
     196        if self.args or self.retann:
     197            self.directive.warn('signature arguments or return annotation '
     198                                'given for autospecmacro %s' % self.fullname)
     199        return ret
    75200
    76201
    … …  
    148273    # TODO: The directive that declares the variable should be the primary (bold) index.
    149274    # TODO: array variables are not handled at all
    150 
    151 
    152 class SpecMacroSourceObject(ObjectDescription):
    153     """
    154     Document a SPEC macro source code file
    155    
    156     This code responds to the ReST file directive::
    157    
    158         .. spec:macrofile:: partial/path/name/somefile.mac
    159             :displayorder: fileorder
    160    
    161     The ``:displayorder`` parameter indicates how the
    162     contents will be sorted for appearance in the ReST document.
    163    
    164         **fileorder**, **file**
    165             Items will be documented in the order in
    166             which they appear in the ``.mac`` file.
    167        
    168         **alphabetical**, **alpha**
    169             Items will be documented in alphabetical order.
    170    
    171     A (near) future enhancement would be to provide for
    172     documenting all macro files in a directory, with optional
    173     recursion into subdirectories.  By default, the code would
    174     only document files that match the glob pattern ``*.mac``.
    175     Such as::
    176    
    177        .. spec:directory:: partial/path/name
    178           :recursion:
    179           :displayorder: alphabetical
    180     """
    181    
    182     # TODO: work-in-progress
    183    
    184     doc_field_types = [
    185         Field('displayorder', label=l_('Display order'), has_arg=False,
    186               names=('displayorder', 'synonym')),
    187     ]
    188 
    189     def add_target_and_index(self, name, sig, signode):
    190         targetname = '%s-%s' % (self.objtype, name)
    191         signode['ids'].append(targetname)
    192         self.state.document.note_explicit_target(signode)
    193         indextext = sig
    194         if indextext:
    195             self.indexnode['entries'].append(('single', indextext,
    196                                               targetname, ''))
    197 
    198     def handle_signature(self, sig, signode):
    199         signode += addnodes.desc_name(sig, sig)
    200         # TODO: this is the place to parse the SPEC macro source code file named in "sig"
    201         '''
    202         Since 2002, SPEC has allowed for triple-quoted strings as extended comments.
    203         Few, if any, have used them.
    204         Assume that they will contain ReST formatted comments.
    205         The first, simplest thing to do is to read the .mac file and only extract
    206         all the extended comments and add them as nodes to the current document.
    207        
    208         An additional step would be to parse for def, cdef, rdef, global, local, const, ...
    209         Another step would be to attach source code and provide links from each to
    210         highlighted source code blocks.
    211         '''
    212         #extended_comments_list = self.parse_macro_file(sig)
    213         view = ViewList([u'TODO: Just handle the macro signature, additional documentation elsewhere'])
    214         #contentnode = nodes.TextElement()
    215         node = nodes.paragraph()
    216         node.document = self.state.document
    217         self.state.nested_parse(view, 0, signode)
    218         # TODO: recognize the ReST formatting in the following extended comment and it needs to be cleaned up
    219         # nodes.TextElement(raw, text)
    220         # sphinx.directives.__init__.py  ObjectDescription.run() method
    221         #  Summary:  This does not belong here, in the signature processing part.
    222         #            Instead, it goes at the directive.run() method.  Where's that here?
    223 #        for extended_comment in extended_comments_list:
    224 #            for line in string2lines(extended_comment):
    225 #                view = ViewList([line])
    226 #                nested_parse_with_titles(self.state, view, signode)
    227         return sig
    228    
    229     def run(self):
    230         # TODO: recognize the ReST formatting in the following extended comment and it needs to be cleaned up
    231         # nodes.TextElement(raw, text)
    232         # sphinx.directives.__init__.py  ObjectDescription.run() method
    233         #  Summary:  This belongs with the directive.run() method.  This is the new place!
    234         #self.content.append(u'')
    235         #self.content.append(u'.. caution:: Use caution.')
    236         from specmacrofileparser import SpecMacrofileParser
    237         macrofile = self.arguments[0]
    238         p = SpecMacrofileParser(macrofile)
    239         return ObjectDescription.run(self)
    240    
    241     def parse_macro_file(self, filename):
    242         """
    243         parse the SPEC macro file and return the ReST blocks
    244        
    245         :param str filename: name (with optional path) of SPEC macro file
    246             (The path is relative to the ``.rst`` document.)
    247         :returns [str]: list of ReST-formatted extended comment blocks (docstrings) from SPEC macro file.
    248        
    249         [future] parse more stuff as planned, this is very simplistic for now
    250         """
    251         results = []
    252         if not os.path.exists(filename):
    253             raise RuntimeError, "could not find: " + filename
    254        
    255         buf = open(filename, 'r').read()
    256         #n = len(buf)
    257         for node in spec_extended_comment_block_sig_re.finditer(buf):
    258             #g = node.group()
    259             #gs = node.groups()
    260             #s = node.start()
    261             #e = node.end()
    262             #t = buf[s:e]
    263             results.append(node.groups()[0])            # TODO: can we get line number also?
    264         return results
    265 
     275    # TODO: variables cited by *role* should link back to their *directive* declarations
    266276
    267277class SpecXRefRole(XRefRole):
    … …  
    311321        'local':      ObjType(l_('local'),      'local'),
    312322        'constant':   ObjType(l_('constant'),   'constant'),
    313         'macrofile':  ObjType(l_('macrofile'),  'macrofile'),
     323        #'specmacro':  ObjType(l_('specmacro'),  'specmacro'),
    314324    }
    315325    directives = {
    … …  
    320330        'local':        SpecVariableObject,
    321331        'constant':     SpecVariableObject,
    322         'macrofile':    SpecMacroSourceObject,
    323332    }
    324333    roles = {
    … …  
    359368def setup(app):
    360369    app.add_domain(SpecDomain)
    361     # http://sphinx.pocoo.org/ext/appapi.html#sphinx.domains.Domain
     370    app.add_autodocumenter(SpecMacroDocumenter)
     371    app.add_config_value('autospecmacrodir_process_subdirs', True, True)

  • specdomain/src/specdomain/sphinxcontrib/specmacrofileparser.py

    r961 r963  
    8484    Parse a SPEC macro file for macro definitions,
    8585    variable declarations, and extended comments.
     86
     87        Since 2002, SPEC has allowed for triple-quoted
     88        strings as extended comments.  Few, if any, have used them.
     89        Assume all extended comments contain ReST formatted comments,
     90        *including initial section titles or transitions*.
     91        The first and simplest thing to do is to read the .mac file and only extract
     92        all the extended comments and add them as nodes to the current document.
     93       
     94        An additional step would be to parse for:
     95        * def
     96        * cdef
     97        * rdef
     98        * global    (done)
     99        * local    (done)
     100        * constant    (done)
     101        * array
     102        * ...
     103       
    86104    '''
    87105
    … …  
    157175                    m['objtype'] = 'extended comment'
    158176                    m['start_line'] = m['end_line'] = line_number
     177                    m['text'] = m['text'].strip()
    159178                    self.findings.append(dict(m))
    160179                    continue
    … …  
    206225        for r in self.findings:
    207226            s.append( '' )
    208             t = '%s %s %d %d %s' % ('*'*20,
     227            t = '%s %s %d %d %s' % ('.. ' + '*'*20,
    209228                                    r['objtype'],
    210229                                    r['start_line'],
    … …  
    212231                                    '*'*20)
    213232            s.append( t )
     233            s.append( '' )
    214234            s.append( r['text'] )
    215235        return '\n'.join(s)

  • specdomain/src/specdomain/test/index.rst

    r927 r963  
    1010
    1111.. toctree::
    12    :maxdepth: 2
     12   :maxdepth: 3
    1313
    1414   test_doc

  • specdomain/src/specdomain/test/test-battery.mac

    r956 r963  
    7474DEPENDENCIES
    7575   Chained macro definitions affected by this macro:
     76   
    7677   - user_precount
    7778   - user_getcounts
    … …  
    193194}'
    194195
    195 """there is more in shutter.mac"""
     196""".. note:: there is much more content in ``shutter.mac``"""
    196197
    197198#==============================================================================

  • specdomain/src/specdomain/test/test_doc.rst

    r953 r963  
    4141   :param str cdef_part: name for this part of the chained macro
    4242   :param str flags: see the manual
    43    
    44 SPEC cdef macro definitions
    45 ++++++++++++++++++++++++++++++++
    4643
    47 .. TODO: pull this subsection once this part is made to work
    48 
    49 There are several different signatures for SPEC's ``cdef`` macro definition.
    50 Here are some examples pulled from the ``SPEC.D`` directory.
    51 
    52 .. note::  At present, the argument list from ``cdef`` macro definitions
    53    is not being parsed or handled.  This will be fixed in a future revision.
    54    
    55 .. literalinclude:: cdef-examples.mac
    56    :linenos:
    57    :language: guess
     44..
     45        SPEC cdef macro definitions
     46        ++++++++++++++++++++++++++++++++
     47       
     48        .. TODO: pull this subsection once this part is made to work
     49       
     50        There are several different signatures for SPEC's ``cdef`` macro definition.
     51        Here are some examples pulled from the ``SPEC.D`` directory.
     52       
     53        .. note::  At present, the argument list from ``cdef`` macro definitions
     54           is not being parsed or handled.  This will be fixed in a future revision.
     55           
     56        .. literalinclude:: cdef-examples.mac
     57           :linenos:
     58           :language: guess
    5859
    5960SPEC Variables
    … …  
    7980+++++++++++++++++++++++
    8081
     82These define the variable.
     83
    8184global variable declaration:
    8285
    83 .. spec:global:: A[]
    84 
    85    ``A[]`` contains the values of all motors
     86        .. spec:global:: A[]
     87       
     88           ``A[]`` contains the values of all motors
    8689
    8790local variable declaration: 
    8891
    89 .. spec:local:: i
    90 
    91    ``i`` is a local loop counter
     92        .. spec:local:: i
     93       
     94           ``i`` is a local loop counter
    9295
    9396array variable declaration:
    9497
    95         tba
     98        *--tba--*
    9699
    97100Variables in Roles
    98101+++++++++++++++++++++++
    99102
    100 * global variable declaration: :spec:global:`A[]`
    101 * local variable declaration:  :spec:local:`i`
    102 * array variable declaration:
    103 * constant declaration:
     103These items should link back to the directives above.
    104104
    105 Python example
    106 ^^^^^^^^^^^^^^
    107 
    108 .. py:function:: python_function(t = None)
    109 
    110    :param t: time_t object or None (defaults to ``now()``)
    111    :type  t: str
     105* global variable declaration:  :spec:global:`A[]`
     106* local variable declaration:   :spec:local:`i`
     107* array variable declaration:   *--tba--*
     108* constant declaration:                 *--tba--*
    112109
    113110
    … …  
    122119^^^^^^^^^^^
    123120
    124 * macro definition: :spec:def:`def_macro`
    125 * function definition: :spec:def:`def_function(arguments)`
     121These items should link back to the directives above.
     122
     123* macro definition:                             :spec:def:`def_macro`
     124* function definition:                          :spec:def:`def_function(arguments)`
    126125* runtime-defined macro definition: :spec:rdef:`rdef_macro`
    127 * chained macro definition: :spec:cdef:`cdef("cdef_macro", "content", "cdef_part", flags)`
     126* chained macro definition:             :spec:cdef:`cdef("cdef_macro", "content", "cdef_part", flags)`
    128127
    129128SPEC Variables
    … …  
    132131The SPEC macro language provides for several types of variable:
    133132
    134 * global variables, such as:  :spec:global:`A[]`
    135 * local variable, such as:  :spec:local:`i`
    136 * array variable declaration:
    137 * constant declaration:
     133* global variables, such as:    :spec:global:`A[]`
     134* local variable, such as:      :spec:local:`i`
     135* array variable declaration:   *--tba--*
     136* constant declaration:                 *--tba--*
    138137
    139138Source code documentation
    140139============================
    141140
    142 Python example
    143 ^^^^^^^^^^^^^^
     141.. toctree::
     142   :maxdepth: 2
    144143
    145 See the python method :py:func:`python_function()` (defined above)
    146 
    147 ..  This should document the Python module supporting the specdomain
    148 
    149 .. py:function:: test.testdoc.radius(x, y)
    150         :noindex:
    151        
    152         :param float x: ordinate
    153         :param float y: abcissa
    154         :returns float: hypotenuse
    155        
    156         return math.sqrt(x*x + y*y)
    157        
    158         The radius function is based on an algorithm of Pythagorus.
    159        
    160         .. note:: The Pythagorean theorem was also cited in the movie *The Wizard of Oz*.
    161 
    162 :class:`SpecVariableObject` (Python Module)
    163 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    164 
    165 .. autoclass:: sphinxcontrib.specdomain.SpecVariableObject
    166     :members:
    167     :undoc-members:
    168     :show-inheritance:
    169 
    170 .. automodule:: sphinxcontrib.specdomain
    171     :members:
    172 
    173 
    174 
    175 SPEC macro source file: ``cdef-examples.mac``
    176 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    177 
    178 .. spec:macrofile:: cdef-examples.mac
    179 
    180 
    181 SPEC macro source file: ``test-battery.mac``
    182 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    183 
    184 .. spec:macrofile:: test-battery.mac
     144   cdef-examples.mac
     145   test-battery.mac
     146   shutter.mac
     147   python-example-source
Note: See TracChangeset for help on using the changeset viewer.