Changeset 3187


Ignore:
Timestamp:
Dec 10, 2017 10:14:04 AM (4 years ago)
Author:
toby
Message:

misc docs cleanups; add 1-ID metadata reader & new config variable

Location:
trunk
Files:
11 edited

Legend:

Unmodified
Added
Removed
  • trunk/GSASIIIO.py

    r3180 r3187  
    3939import sys
    4040import re
     41import glob
    4142import random as ran
    4243import GSASIIpath
     
    363364
    364365def GetImageData(G2frame,imagefile,imageOnly=False,ImageTag=None,FormatName=''):
    365     '''Read a single image with an image importer.
     366    '''Read a single image with an image importer. This is called to reread an image
     367    after it has already been imported with :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`
     368    (or :func:`ReadImages` in Auto Integration) so it is not necessary to reload metadata.
    366369
    367370    :param wx.Frame G2frame: main GSAS-II Frame and data object.
     
    439442    using image importers. Called only in :meth:`AutoIntFrame.OnTimerLoop`.
    440443
     444    ToDo: Images are most commonly read in :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`
     445    which is called from :meth:`GSASIIdataGUI.GSASII.OnImportImage`
     446    it would be good if these routines used a common code core so that changes need to
     447    be made in only one place.
     448
    441449    :param wx.Frame G2frame: main GSAS-II Frame and data object.
    442450    :param str imagefile: name of image file
     
    492500                    rd.Image = rd.Image.T
    493501                rd.Data['ImageTag'] = rd.repeatcount
     502                if GSASIIpath.GetConfigValue('Image_1IDmetadata'):
     503                    rd.readfilename = imagefile
     504                    Get1IDMetadata(rd)
    494505                LoadImage2Tree(imagefile,G2frame,rd.Comments,rd.Data,rd.Npix,rd.Image)
    495506                repeat = rd.repeat
     
    21352146    return Layer
    21362147
     2148def Read1IDpar(imagefile):
     2149    '''Reads image metadata from a 1-ID style .par file from the same directory
     2150    as the image. The .par file has any number of columns separated by spaces.
     2151    As an index to the .par file a second label file must be specified with the
     2152    same file name as the .par file but the extension must be .XXX_lbls (where
     2153    .XXX is the extension of the image) or if that is not present extension
     2154    .lbls
     2155
     2156    :param str imagefile: the full name of the image file (with directory and extension)
     2157
     2158    :returns: a dict with parameter values. Named parameters will have the type based on
     2159       the specified Python function, named columns will be character strings
     2160   
     2161    The contents of the label file will look like this::
     2162   
     2163        # define keywords
     2164        filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34
     2165        distance: float | 75
     2166        wavelength:lambda keV: 12.398425/float(keV)|9
     2167        pixelSize:lambda x: [74.8, 74.8]|0
     2168        ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4
     2169        # define other variables
     2170        0:day
     2171        1:month
     2172        2:date
     2173        3:time
     2174        4:year
     2175        7:I_ring
     2176
     2177    This file contains three types of lines in any order.
     2178     * Named parameters are evaluated with user-supplied Python code (see
     2179       subsequent information). Specific named parameters are used
     2180       to determine values that are used for image interpretation (see table,
     2181       below). Any others are copied to the Comments subsection of the Image
     2182       tree item.
     2183     * Column labels are defined with a column number (integer) followed by
     2184       a colon (:) and a label to be assigned to that column. All labeled
     2185       columns are copied to the Image's Comments subsection.
     2186     * Comments are any line that does not contain a colon.
     2187
     2188    Note that columns are numbered starting at zero.
     2189
     2190    Any named parameter may be defined provided it is not a valid integer,
     2191    but the named parameters in the table have special meanings, as descibed.
     2192    The parameter name is followed by a colon. After the colon, specify
     2193    Python code that defines or specifies a function that will be called to
     2194    generate a value for that parameter. After that code, supply a vertical
     2195    bar (|) and then a list of one more more columns that will be supplied
     2196    as arguments to that function. The examples above are discussed below:
     2197
     2198    ``filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34``
     2199        Here the function to be used is defined with a lambda statement:
     2200       
     2201          lambda x,y: "{}_{:0>6}".format(x,y)
     2202
     2203        This function will use the format function to create a file name from the
     2204        contents of columns 33 and 34. The first parameter (x, col. 33) is inserted directly into
     2205        the file name, followed by a underscore (_), followed by the second parameter (y, col. 34),
     2206        which will be left-padded with zeros to six characters (format directive ``:0>6``).
     2207       
     2208    ``distance: float | 75``
     2209        Here the contents of column 75 will be converted to a floating point number
     2210        by calling float on it. Note that the spaces here are ignored.
     2211       
     2212    ``wavelength:lambda keV: 12.398425/float(keV)|9``
     2213        Here we define an algebraic expression to convert an energy in keV to a
     2214        wavelength and pass the contents of column 9 as that input energy
     2215       
     2216    ``pixelSize:lambda x: [74.8, 74.8]|0``
     2217        In this case the pixel size is a constant (a list of two numbers). The first
     2218        column is passed as an argument as at least one argument is required, but that
     2219        value is not used in the expression.
     2220
     2221     ``ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4``
     2222        This example defines a parameter that takes items in the first five columns
     2223        and formats them in a different way. This parameter is not one of the pre-defined
     2224        parameter names below. Some external code could be used to change the month string
     2225        (argument ``m``) to a integer from 1 to 12.
     2226           
     2227    **Pre-defined parameter names**
     2228   
     2229    =============  =========  ========  ====================================================
     2230     keyword       required    type      Description
     2231    =============  =========  ========  ====================================================
     2232       filename    yes         str      generates the file name prefix for the matching image
     2233                                        file (MyImage001 for file /tmp/MyImage001.tif).
     2234     polarization  no         float     generates the polarization expected based on the
     2235                                        monochromator angle, defaults to 0.99.
     2236       center      no         list of   generates the approximate beam center on the detector
     2237                              2 floats  in mm, such as [204.8, 204.8].
     2238       distance    yes        float     generates the distance from the sample to the detector
     2239                                        in mm
     2240       pixelSize   no         list of   generates the size of the pixels in microns such as
     2241                              2 floats  [200.0, 200.0].
     2242       wavelength  yes        float     generates the wavelength in Angstroms
     2243    =============  =========  ========  ====================================================
     2244
     2245   
     2246    '''
     2247    dir,fil = os.path.split(os.path.abspath(imagefile))
     2248    imageName,ext = os.path.splitext(fil)
     2249    parfiles = glob.glob(os.path.join(dir,'*.par'))
     2250    if len(parfiles) == 0:
     2251        print('Sorry, No 1-ID .par file found in '+dir)
     2252        return {}
     2253    for parFil in parfiles: # loop over all .par files (hope just 1) in image dir until image is found
     2254        parRoot = os.path.splitext(parFil)[0]
     2255        for e in (ext+'_lbls','.lbls'):
     2256            if os.path.exists(parRoot+e):
     2257                lblFil = parRoot+e
     2258                break
     2259        else:
     2260            print('Warning: No labels definitions found for '+parFil)
     2261            return {}
     2262        fp = open(lblFil,'Ur')         # read column labels
     2263        lbldict = {}
     2264        fmt = ""
     2265        keyExp = {}
     2266        keyCols = {}
     2267        for line in fp: # read label definitions
     2268            items = line.strip().split(':')
     2269            if len(items) < 2: continue # no colon, line is a comment
     2270            # is this line a definition for a named parameter?
     2271            key = items[0]
     2272            try:
     2273                int(key)
     2274            except ValueError: # named parameters are not valid numbers
     2275                items = line.split(':',1)[1].split('|')
     2276                try:
     2277                    keyExp[key] = eval(items[0]) # compile the expression
     2278                    if not callable(keyExp[key]):
     2279                        raise Exception("Expression not a function")
     2280                except Exception as msg:
     2281                    print('Warning: Expression "{}" is not valid for key {}'.format(items[0],key))
     2282                    print(msg)
     2283                keyCols[key] = [int(i) for i in items[1].strip().split(',')]
     2284                continue
     2285            if len(items) == 2: # simple column definition
     2286                lbldict[int(items[0])] = items[1]
     2287        fp.close()
     2288        if 'filename' not in keyExp:
     2289            print("Warning: No valid filename expression in {} file, skipping .par".format(lblFil))
     2290            continue
     2291        # read the .par, looking for the matching image rootname
     2292        fp = open(parFil,'Ur')
     2293        for i,line in enumerate(fp):
     2294            items = line.strip().split(' ')
     2295            name = keyExp['filename'](*[items[j] for j in keyCols['filename']])
     2296            if name == imageName: # got our match, parse the line and finish
     2297                metadata = {lbldict[i]:items[i] for i in lbldict}
     2298                metadata.update(
     2299                    {key:keyExp[key](*[items[j] for j in keyCols[key]]) for key in keyExp})
     2300                metadata['par file'] = parFil
     2301                metadata['lbls file'] = lblFil
     2302                print("Metadata read from {}".format(parFil))
     2303                return metadata
     2304        else:
     2305            print("Image {} not found in {}".format(imageName,parFil))
     2306            fp.close()
     2307            continue
     2308        fp.close()
     2309    else:
     2310        print("Warning: No 1-ID metadata for image {}".format(imageName))
     2311        return {}
     2312
     2313def Get1IDMetadata(reader):
     2314    '''Add 1-ID metadata to an image using a 1-ID .par file using :func:`Read1IDpar`
     2315   
     2316    :param reader: a reader object from reading an image
     2317   
     2318    '''
     2319    parParms = Read1IDpar(reader.readfilename)
     2320    if not parParms: return # check for read failure
     2321    specialKeys = ('filename',"polarization", "center", "distance", "pixelSize", "wavelength",)
     2322    reader.Comments = ['Metadata from {} assigned by {}'.format(parParms['par file'],parParms['lbls file'])]
     2323    for key in parParms:
     2324        if key in specialKeys+('par file','lbls file'): continue
     2325        reader.Comments += ["{} = {}".format(key,parParms[key])]
     2326    if "polarization" in parParms:
     2327        reader.Data['PolaVal'][0] = parParms["polarization"]
     2328    else:
     2329        reader.Data['PolaVal'][0] = 0.99
     2330    if "center" in parParms:
     2331        reader.Data['center'] = parParms["center"]
     2332    if "pixelSize" in parParms:
     2333        reader.Data['pixelSize'] = parParms["pixelSize"]
     2334    if "wavelength" in parParms:
     2335        reader.Data['wavelength'] = parParms['wavelength']
     2336    else:
     2337        print('Error: wavelength not defined in {}'.format(parParms['lbls file']))
     2338    if "distance" in parParms:
     2339        reader.Data['distance'] = parParms['distance']
     2340        reader.Data['setdist'] = parParms['distance']
     2341    else:
     2342        print('Error: distance not defined in {}'.format(parParms['lbls file']))
     2343
    21372344if __name__ == '__main__':
    21382345    import GSASIIdataGUI
  • trunk/GSASIIctrlGUI.py

    r3167 r3187  
    45594559    #['ExampleDir', 'ExamplePage.html', 'Example Tutorial Title'],
    45604560    )
     4561'''A catalog of GSAS-II tutorials with headings. This is the master list of GSAS-II tutorials and must be updated when tutorials are
     4562added. Each item has either one or three items.
     4563Titles are single item in a list or tuple. Tutorials have three items: (a) the name of the directory,
     4564(b) the name of the web page and (c) a title for the tutorial.
     4565Tutorials that depend on a previous tutorial being completed should have the title for
     4566the tutorial indented by five spaces.
     4567
     4568Note that :data:`tutorialCatalog` is generated from this tuple.
     4569Also see :mod:`makeTutorial` which is used to read this and create a web page.
     4570'''
     4571
     4572#A catalog of GSAS-II tutorials generated from the table in :data:`tutorialIndex`
    45614573tutorialCatalog = [l for l in tutorialIndex if len(l) == 3]
    45624574
  • trunk/GSASIIdataGUI.py

    r3185 r3187  
    783783                            if rd.sumfile:
    784784                                rd.readfilename = rd.sumfile
     785                            if GSASIIpath.GetConfigValue('Image_1IDmetadata'):
     786                                G2IO.Get1IDMetadata(rd)
    785787                            G2IO.LoadImage2Tree(rd.readfilename,self,rd.Comments,rd.Data,rd.Npix,rd.Image)
    786788                            rd_list.append(True) # save a stub the result before it is written over
  • trunk/GSASIImath.py

    r3164 r3187  
    8888
    8989    References:
    90     .. [1] G. Strang, *Linear Algebra and Its Applications*, 2nd Ed., Orlando,
    91            FL, Academic Press, Inc., 1980, pp. 139-142.
     90    .. [1] G. Strang, *Linear Algebra and Its Applications*, 2nd Ed., Orlando, FL, Academic Press, Inc., 1980, pp. 139-142.
    9291
    9392    """
     
    874873    :param array Y: normalized vector
    875874   
    876     :returns: array M: transformation matrix
     875    :returns: array M: transformation matrix
     876   
    877877    use as XYZ' = np.inner(M,XYZ) where XYZ are Cartesian
    878878   
     
    41274127           lower=-100, upper=100, dwell=50, slope=0.9,ranStart=False,
    41284128           ranRange=0.10,autoRan=False,dlg=None):
    4129     ''' Scipy license:
    4130         Copyright (c) 2001, 2002 Enthought, Inc.
    4131     All rights reserved.
    4132    
    4133     Copyright (c) 2003-2016 SciPy Developers.
    4134     All rights reserved.
    4135    
    4136     Redistribution and use in source and binary forms, with or without
    4137     modification, are permitted provided that the following conditions are met:
    4138    
    4139       a. Redistributions of source code must retain the above copyright notice,
    4140          this list of conditions and the following disclaimer.
    4141       b. Redistributions in binary form must reproduce the above copyright
    4142          notice, this list of conditions and the following disclaimer in the
    4143          documentation and/or other materials provided with the distribution.
    4144       c. Neither the name of Enthought nor the names of the SciPy Developers
    4145          may be used to endorse or promote products derived from this software
    4146          without specific prior written permission.
    4147     '''
    4148 
    41494129    """Minimize a function using simulated annealing.
    41504130
     
    42384218        T_new = T0 * exp(-c * k**quench)
    42394219
    4240 
    42414220    """
     4221   
     4222    ''' Scipy license:
     4223        Copyright (c) 2001, 2002 Enthought, Inc.
     4224    All rights reserved.
     4225   
     4226    Copyright (c) 2003-2016 SciPy Developers.
     4227    All rights reserved.
     4228   
     4229    Redistribution and use in source and binary forms, with or without
     4230    modification, are permitted provided that the following conditions are met:
     4231   
     4232      a. Redistributions of source code must retain the above copyright notice,
     4233         this list of conditions and the following disclaimer.
     4234      b. Redistributions in binary form must reproduce the above copyright
     4235         notice, this list of conditions and the following disclaimer in the
     4236         documentation and/or other materials provided with the distribution.
     4237      c. Neither the name of Enthought nor the names of the SciPy Developers
     4238         may be used to endorse or promote products derived from this software
     4239         without specific prior written permission.
     4240    '''
    42424241    x0 = asarray(x0)
    42434242    lower = asarray(lower)
  • trunk/GSASIIobj.py

    r3151 r3187  
    27382738       tim0.start('in section 1')
    27392739       tim0.start('in section 2')
     2740       
    27402741    etc. (Note that each section should have a unique label.)
    27412742
  • trunk/GSASIIscriptable.py

    r3157 r3187  
    99########### SVN repository information ###################
    1010"""
    11 *GSASIIscriptable: Scripting Tools*
    12 ======================================
     11*GSASIIscriptable: Scripting Interface*
     12=======================================
    1313
    1414Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
     
    312312import datetime as dt
    313313import sys
    314 import cPickle
     314import platform
     315if '2' in platform.python_version_tuple()[0]:
     316    import cPickle
     317else:
     318    import _pickle as cPickle
    315319import imp
    316320import copy
     
    13261330        if outputnames:
    13271331            if len(refinements) != len(outputnames):
    1328                 raise ValueError("Should have same number of outuputs to"
     1332                raise ValueError("Should have same number of outputs to"
    13291333                                 "refinements")
    13301334        else:
  • trunk/config_example.py

    r3170 r3187  
    170170Show_timing = False
    171171'''If True, shows various timing results.'''
     172
     173Image_1IDmetadata = False
     174'''When True, when images are read, GSAS-II will read metadata from a 1-ID style
     175.par and .lbls (or .lbls_EXT) file.
     176'''
  • trunk/docs/source/GSASIIscripts.rst

    r3102 r3187  
    1 .. automodule:: GSASIIscriptable
    2     :members:
    3 
    41*GSAS-II Misc Scripts*
    52======================
     
    2017    :members:
    2118
     19.. automodule:: makeLinux
     20    :members:
     21
     22.. automodule:: makeTutorial
     23    :members:
     24
    2225.. automodule:: unit_tests
    2326    :members:
  • trunk/docs/source/GSASIIutil.rst

    r2027 r3187  
    11*GSAS-II Utility Modules*
    22=========================
    3 
    4 .. automodule:: GSASIIdata
    5     :members:
    63
    74.. automodule:: GSASIIpath
     
    2118
    2219.. automodule:: GSASIIspc
     20    :members:
     21
     22.. automodule:: GSASIIdata
     23    :members:
     24
     25.. automodule:: GSASIIfiles
     26    :members:
     27
     28.. automodule:: GSASIImpsubs
    2329    :members:
    2430
  • trunk/docs/source/index.rst

    r3000 r3187  
    2020  GSASIIpwd.rst
    2121  SAS.rst
     22  GSASIIscriptable.rst
    2223  GSASIIscripts.rst
    2324  exports.rst
  • trunk/makeTutorial.py

    r3135 r3187  
    1 # create an HTML page from the tutorials in GSASIIctrlGUI.tutorialCatalog
     1# -*- coding: utf-8 -*-
     2#makeTutorial.py
     3########### SVN repository information ###################
     4# $Date: $
     5# $Author: toby $
     6# $Revision: 3167 $
     7# $URL: $
     8# $Id: $
     9########### SVN repository information ###################
     10'''
     11*makeTutorial: Make Tutorial Web Page*
     12---------------------------------------------
     13
     14Creates an HTML page (``GSASII/help/Tutorials.html``) listing all the tutorials in
     15:data:`GSASIIctrlGUI.tutorialIndex`. Run this after adding a new tutorial to that
     16index.
     17'''
    218
    319from __future__ import print_function
Note: See TracChangeset for help on using the changeset viewer.