Changeset 3814


Ignore:
Timestamp:
Feb 10, 2019 4:40:06 PM (5 years ago)
Author:
toby
Message:

refactor to move some IO-only routines; add initial image support to scriptable

Location:
trunk
Files:
7 edited

Legend:

Unmodified
Added
Removed

  • trunk/GSASIIIO.py

    r3810 r3814  
    6161import GSASIIstrIO as G2stIO
    6262import GSASIImapvars as G2mv
     63import GSASIIfiles as G2fil
    6364try:
    6465    import GSASIIctrlGUI as G2G
    … …  
    507508                rd.readfilename = imagefile
    508509                # Load generic metadata, as configured
    509                 GetColumnMetadata(rd)
     510                G2fil.GetColumnMetadata(rd)
    510511                LoadImage2Tree(imagefile,G2frame,rd.Comments,rd.Data,rd.Npix,rd.Image)
    511512                repeat = rd.repeat
    … …  
    22392240    return Layer
    22402241
    2241 def readColMetadata(imagefile):
    2242     '''Reads image metadata from a column-oriented metadata table
    2243     (1-ID style .par file). Called by :func:`GetColumnMetadata`
    2244    
    2245     The .par file has any number of columns separated by spaces.
    2246     The directory for the file must be specified in
    2247     Config variable ``Column_Metadata_directory``.
    2248     As an index to the .par file a second "label file" must be specified with the
    2249     same file root name as the .par file but the extension must be .XXX_lbls (where
    2250     .XXX is the extension of the image) or if that is not present extension
    2251     .lbls.
    2252 
    2253     :param str imagefile: the full name of the image file (with extension, directory optional)
    2254 
    2255     :returns: a dict with parameter values. Named parameters will have the type based on
    2256        the specified Python function, named columns will be character strings
    2257    
    2258     The contents of the label file will look like this::
    2259    
    2260         # define keywords
    2261         filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34
    2262         distance: float | 75
    2263         wavelength:lambda keV: 12.398425/float(keV)|9
    2264         pixelSize:lambda x: [74.8, 74.8]|0
    2265         ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4
    2266         Temperature: float|53
    2267         FreePrm2: int | 34 | Free Parm2 Label
    2268         # define other variables
    2269         0:day
    2270         1:month
    2271         2:date
    2272         3:time
    2273         4:year
    2274         7:I_ring
    2275 
    2276     This file contains three types of lines in any order.
    2277      * Named parameters are evaluated with user-supplied Python code (see
    2278        subsequent information). Specific named parameters are used
    2279        to determine values that are used for image interpretation (see table,
    2280        below). Any others are copied to the Comments subsection of the Image
    2281        tree item.
    2282      * Column labels are defined with a column number (integer) followed by
    2283        a colon (:) and a label to be assigned to that column. All labeled
    2284        columns are copied to the Image's Comments subsection.
    2285      * Comments are any line that does not contain a colon.
    2286 
    2287     Note that columns are numbered starting at zero.
    2288 
    2289     Any named parameter may be defined provided it is not a valid integer,
    2290     but the named parameters in the table have special meanings, as descibed.
    2291     The parameter name is followed by a colon. After the colon, specify
    2292     Python code that defines or specifies a function that will be called to
    2293     generate a value for that parameter.
    2294 
    2295     Note that several keywords, if defined in the Comments, will be found and
    2296     placed in the appropriate section of the powder histogram(s)'s Sample
    2297     Parameters after an integration: ``Temperature``,``Pressure``,``Time``,
    2298     ``FreePrm1``,``FreePrm2``,``FreePrm3``,``Omega``,``Chi``, and ``Phi``.
    2299 
    2300     After the Python code, supply a vertical bar (|) and then a list of one
    2301     more more columns that will be supplied as arguments to that function.
    2302 
    2303     Note that the labels for the three FreePrm items can be changed by
    2304     including that label as a third item with an additional vertical bar. Labels
    2305     will be ignored for any other named parameters.
    2306    
    2307     The examples above are discussed here:
    2308 
    2309     ``filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34``
    2310         Here the function to be used is defined with a lambda statement::
    2311        
    2312           lambda x,y: "{}_{:0>6}".format(x,y)
    2313 
    2314         This function will use the format function to create a file name from the
    2315         contents of columns 33 and 34. The first parameter (x, col. 33) is inserted directly into
    2316         the file name, followed by a underscore (_), followed by the second parameter (y, col. 34),
    2317         which will be left-padded with zeros to six characters (format directive ``:0>6``).
    2318 
    2319         When there will be more than one image generated per line in the .par file, an alternate way to
    2320         generate list of file names takes into account the number of images generated::
    2321 
    2322           lambda x,y,z: ["{}_{:0>6}".format(x,int(y)+i) for i in range(int(z))]
    2323 
    2324         Here a third parameter is used to specify the number of images generated, where
    2325         the image number is incremented for each image.
    2326          
    2327     ``distance: float | 75``
    2328         Here the contents of column 75 will be converted to a floating point number
    2329         by calling float on it. Note that the spaces here are ignored.
    2330        
    2331     ``wavelength:lambda keV: 12.398425/float(keV)|9``
    2332         Here we define an algebraic expression to convert an energy in keV to a
    2333         wavelength and pass the contents of column 9 as that input energy
    2334        
    2335     ``pixelSize:lambda x: [74.8, 74.8]|0``
    2336         In this case the pixel size is a constant (a list of two numbers). The first
    2337         column is passed as an argument as at least one argument is required, but that
    2338         value is not used in the expression.
    2339 
    2340     ``ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4``
    2341         This example defines a parameter that takes items in the first five columns
    2342         and formats them in a different way. This parameter is not one of the pre-defined
    2343         parameter names below. Some external code could be used to change the month string
    2344         (argument ``m``) to a integer from 1 to 12.
    2345        
    2346     ``FreePrm2: int | 34 | Free Parm2 Label``
    2347         In this example, the contents of column 34 will be converted to an integer and
    2348         placed as the second free-named parameter in the Sample Parameters after an
    2349         integration. The label for this parameter will be changed to "Free Parm2 Label".
    2350            
    2351     **Pre-defined parameter names**
    2352    
    2353     =============  =========  ========  =====================================================
    2354      keyword       required    type      Description
    2355     =============  =========  ========  =====================================================
    2356        filename    yes         str or   generates the file name prefix for the matching image
    2357                                list     file (MyImage001 for file /tmp/MyImage001.tif) or
    2358                                         a list of file names.
    2359      polarization  no         float     generates the polarization expected based on the
    2360                                         monochromator angle, defaults to 0.99.
    2361        center      no         list of   generates the approximate beam center on the detector
    2362                               2 floats  in mm, such as [204.8, 204.8].
    2363        distance    yes        float     generates the distance from the sample to the detector
    2364                                         in mm
    2365        pixelSize   no         list of   generates the size of the pixels in microns such as
    2366                               2 floats  [200.0, 200.0].
    2367        wavelength  yes        float     generates the wavelength in Angstroms
    2368     =============  =========  ========  =====================================================
    2369    
    2370     '''
    2371     dir,fil = os.path.split(os.path.abspath(imagefile))
    2372     imageName,ext = os.path.splitext(fil)
    2373     if not GSASIIpath.GetConfigValue('Column_Metadata_directory'): return
    2374     parfiles = glob.glob(os.path.join(GSASIIpath.GetConfigValue('Column_Metadata_directory'),'*.par'))
    2375     if len(parfiles) == 0:
    2376         print('Sorry, No Column metadata (.par) file found in '+
    2377               GSASIIpath.GetConfigValue('Column_Metadata_directory'))
    2378         return {}
    2379     for parFil in parfiles: # loop over all .par files (hope just 1) in image dir until image is found
    2380         parRoot = os.path.splitext(parFil)[0]
    2381         for e in (ext+'_lbls','.lbls'):
    2382             if os.path.exists(parRoot+e):
    2383                 lblFil = parRoot+e
    2384                 break
    2385         else:
    2386             print('Warning: No labels definitions found for '+parFil)
    2387             continue
    2388         labels,lbldict,keyCols,keyExp,errors = readColMetadataLabels(lblFil)
    2389         if errors:
    2390             print('Errors in labels file '+lblFil)
    2391             for i in errors: print('  '+i)
    2392             continue
    2393         else:
    2394             print('Read '+lblFil)
    2395         # scan through each line in this .par file, looking for the matching image rootname
    2396         fp = open(parFil,'Ur')
    2397         for iline,line in enumerate(fp):
    2398             items = line.strip().split(' ')
    2399             nameList = keyExp['filename'](*[items[j] for j in keyCols['filename']])
    2400             if type(nameList) is str:
    2401                 if nameList != imageName: continue
    2402                 name = nameList
    2403             else:
    2404                 for name in nameList:
    2405                     if name == imageName: break # got a match
    2406                 else:
    2407                     continue
    2408             # parse the line and finish
    2409             metadata = evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp)
    2410             metadata['par file'] = parFil
    2411             metadata['lbls file'] = lblFil
    2412             print("Metadata read from {} line {}".format(parFil,iline+1))
    2413             fp.close()
    2414             return metadata
    2415         else:
    2416             print("Image {} not found in {}".format(imageName,parFil))
    2417             fp.close()
    2418             continue
    2419         fp.close()
    2420     else:
    2421         print("Warning: No .par metadata for image {}".format(imageName))
    2422         return {}
    2423 
    2424 def readColMetadataLabels(lblFil):
    2425     '''Read the .*lbls file and setup for metadata assignments
    2426     '''
    2427     lbldict = {}
    2428     keyExp = {}
    2429     keyCols = {}
    2430     labels = {}
    2431     errors = []
    2432     fp = open(lblFil,'Ur')         # read column labels
    2433     for iline,line in enumerate(fp): # read label definitions
    2434         line = line.strip()
    2435         if not line or line[0] == '#': continue # comments
    2436         items = line.split(':')
    2437         if len(items) < 2: continue # lines with no colon are also comments
    2438         # does this line a definition for a named parameter?
    2439         key = items[0]
    2440         try:
    2441             int(key)
    2442         except ValueError: # try as named parameter since not a valid number
    2443             items = line.split(':',1)[1].split('|')
    2444             try:
    2445                 f = eval(items[0]) # compile the expression
    2446                 if not callable(f):
    2447                     errors += ['Expression "{}" for key {} is not a function (line {})'.
    2448                            format(items[0],key,iline)]
    2449                     continue
    2450                 keyExp[key] = f
    2451             except Exception as msg:
    2452                 errors += ['Expression "{}" for key {} is not valid (line {})'.
    2453                            format(items[0],key,iline)]
    2454                 errors += [str(msg)]
    2455                 continue
    2456             keyCols[key] = [int(i) for i in items[1].strip().split(',')]
    2457             if key.lower().startswith('freeprm') and len(items) > 2:
    2458                 labels[key] = items[2]
    2459             continue
    2460         if len(items) == 2: # simple column definition
    2461             lbldict[int(items[0])] = items[1]
    2462     fp.close()
    2463     if 'filename' not in keyExp:
    2464         errors += ["File {} is invalid. No valid filename expression.".format(lblFil)]
    2465     return labels,lbldict,keyCols,keyExp,errors
    2466 
    2467 def evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp,ShowError=False):
    2468     '''Evaluate the metadata for a line in the .par file
    2469     '''
    2470     metadata = {lbldict[j]:items[j] for j in lbldict}
    2471     named = {}
    2472     for key in keyExp:
    2473         try:
    2474             res = keyExp[key](*[items[j] for j in keyCols[key]])
    2475         except:
    2476             if ShowError:
    2477                 res = "*** error ***"
    2478             else:
    2479                 continue
    2480         named[key] = res
    2481     metadata.update(named)
    2482     for lbl in labels: # add labels for FreePrm's
    2483         metadata['label_'+lbl[4:].lower()] = labels[lbl]
    2484     return metadata
    2485 
    2486 def GetColumnMetadata(reader):
    2487     '''Add metadata to an image from a column-type metadata file
    2488     using :func:`readColMetadata`
    2489    
    2490     :param reader: a reader object from reading an image
    2491    
    2492     '''
    2493     if not GSASIIpath.GetConfigValue('Column_Metadata_directory'): return
    2494     parParms = readColMetadata(reader.readfilename)
    2495     if not parParms: return # check for read failure
    2496     specialKeys = ('filename',"polarization", "center", "distance", "pixelSize", "wavelength",)
    2497     reader.Comments = ['Metadata from {} assigned by {}'.format(parParms['par file'],parParms['lbls file'])]
    2498     for key in parParms:
    2499         if key in specialKeys+('par file','lbls file'): continue
    2500         reader.Comments += ["{} = {}".format(key,parParms[key])]
    2501     if "polarization" in parParms:
    2502         reader.Data['PolaVal'][0] = parParms["polarization"]
    2503     else:
    2504         reader.Data['PolaVal'][0] = 0.99
    2505     if "center" in parParms:
    2506         reader.Data['center'] = parParms["center"]
    2507     if "pixelSize" in parParms:
    2508         reader.Data['pixelSize'] = parParms["pixelSize"]
    2509     if "wavelength" in parParms:
    2510         reader.Data['wavelength'] = parParms['wavelength']
    2511     else:
    2512         print('Error: wavelength not defined in {}'.format(parParms['lbls file']))
    2513     if "distance" in parParms:
    2514         reader.Data['distance'] = parParms['distance']
    2515         reader.Data['setdist'] = parParms['distance']
    2516     else:
    2517         print('Error: distance not defined in {}'.format(parParms['lbls file']))
    2518 
    2519 def testColumnMetadata(G2frame):
    2520     '''Test the column-oriented metadata parsing, as implemented at 1-ID, by showing results
    2521     when using a .par and .lbls pair.
    2522    
    2523      * Select a .par file, if more than one in selected dir.
    2524      * Select the .*lbls file, if more than one matching .par file.
    2525      * Parse the .lbls file, showing errors if encountered; loop until errors are fixed.
    2526      * Search for an image or a line in the .par file and show the results when interpreted
    2527      
    2528     See :func:`readColMetadata` for more details.
    2529     '''
    2530     if not GSASIIpath.GetConfigValue('Column_Metadata_directory'):
    2531         G2G.G2MessageBox(G2frame,'The configuration option for I-ID Metadata is not set.\n'+
    2532                          'Please use the File/Preferences menu to set Column_Metadata_directory',
    2533                          'Warning')
    2534         return
    2535     parFiles = glob.glob(os.path.join(GSASIIpath.GetConfigValue('Column_Metadata_directory'),'*.par'))
    2536     if not parFiles:
    2537         G2G.G2MessageBox(G2frame,'No .par files found in directory {}. '
    2538                          .format(GSASIIpath.GetConfigValue('Column_Metadata_directory'))+
    2539                          '\nThis is set by config variable Column_Metadata_directory '+
    2540                          '(Set in File/Preferences menu).',
    2541                          'Warning')
    2542         return
    2543     parList = []
    2544     for parFile in parFiles:
    2545         lblList = []
    2546         parRoot = os.path.splitext(parFile)[0]
    2547         for f in glob.glob(parRoot+'.*lbls'):
    2548             if os.path.exists(f): lblList.append(f)
    2549         if not len(lblList):
    2550             continue
    2551         parList.append(parFile)
    2552     if len(parList) == 0:
    2553         G2G.G2MessageBox(G2frame,'No .lbls or .EXT_lbls file found for .par file(s) in directory {}. '
    2554                          .format(GSASIIpath.GetConfigValue('Column_Metadata_directory'))+
    2555                          '\nThis is set by config variable Column_Metadata_directory '+
    2556                          '(Set in File/Preferences menu).',
    2557                          'Warning')
    2558         return
    2559     elif len(parList) == 1:
    2560         parFile = parList[0]
    2561     else:
    2562         dlg = G2G.G2SingleChoiceDialog(G2frame,
    2563                 'More than 1 .par file found. (Better if only 1!). Choose the one to test in '+
    2564                 GSASIIpath.GetConfigValue('Column_Metadata_directory'),
    2565                 'Choose .par file', [os.path.split(i)[1] for i in parList])
    2566         if dlg.ShowModal() == wx.ID_OK:
    2567             parFile = parList[dlg.GetSelection()]
    2568             dlg.Destroy()
    2569         else:
    2570             dlg.Destroy()
    2571             return
    2572     # got .par file; now work on .*lbls file
    2573     lblList = []
    2574     parRoot = os.path.splitext(parFile)[0]
    2575     for f in glob.glob(parRoot+'.*lbls'):
    2576         if os.path.exists(f): lblList.append(f)
    2577     if not len(lblList):
    2578         raise Exception('How did this happen! No .*lbls files for '+parFile)
    2579     elif len(lblList) == 1:
    2580         lblFile = lblList[0]
    2581     else:
    2582         dlg = G2G.G2SingleChoiceDialog(G2frame,
    2583                 'Select label file for .par file '+parFile,
    2584                 'Choose label file', [os.path.split(i)[1] for i in lblList])
    2585         if dlg.ShowModal() == wx.ID_OK:
    2586             lblFile = lblList[dlg.GetSelection()]
    2587             dlg.Destroy()
    2588         else:
    2589             dlg.Destroy()
    2590             return
    2591     # parse the labels file
    2592     errors = True
    2593     while errors:
    2594         labels,lbldict,keyCols,keyExp,errors = readColMetadataLabels(lblFile)
    2595         if errors:
    2596             t = "Error reading file "+lblFile
    2597             for l in errors:
    2598                 t += '\n'
    2599                 t += l
    2600             t += "\n\nPlease edit the file and press OK (or Cancel to quit)"
    2601             dlg = wx.MessageDialog(G2frame,message=t,
    2602                 caption="Read Error",style=wx.ICON_ERROR| wx.OK|wx.STAY_ON_TOP|wx.CANCEL)
    2603             if dlg.ShowModal() != wx.ID_OK: return           
    2604     # request a line number, read that line
    2605     dlg = G2G.SingleStringDialog(G2frame,'Read what',
    2606                                  'Enter a line number or an image file name (-1=last line)',
    2607                                  '-1',size=(400,-1))
    2608     if dlg.Show():
    2609         fileorline = dlg.GetValue()
    2610         dlg.Destroy()
    2611     else:
    2612         dlg.Destroy()
    2613         return
    2614     # and report the generated key pairs in metadata dict
    2615     linenum = None
    2616     try:
    2617         linenum = int(fileorline)
    2618     except:
    2619         imageName = os.path.splitext(os.path.split(fileorline)[1])[0]
    2620 
    2621     fp = open(parFile,'Ur')
    2622     for iline,line in enumerate(fp):
    2623         if linenum is not None:
    2624             if iline == linenum:
    2625                 items = line.strip().split(' ')
    2626                 n = "Line {}".format(iline)
    2627                 break
    2628             else:
    2629                 continue
    2630         else:
    2631             items = line.strip().split(' ')
    2632             nameList = keyExp['filename'](*[items[j] for j in keyCols['filename']])
    2633             if type(nameList) is str:
    2634                 if nameList != imageName: continue
    2635                 name = nameList
    2636                 break
    2637             else:
    2638                 for name in nameList:
    2639                     print (name,name == imageName)
    2640                     if name == imageName:
    2641                         n = "Image {} found in line {}".format(imageName,iline)
    2642                         break # got a match
    2643                 else:
    2644                     continue
    2645                 break
    2646     else:
    2647         if linenum is not None:
    2648             n = "Line {}".format(iline)
    2649         else:
    2650             n = "Image {} not found. Reporting line {}".format(imageName,iline)
    2651         items = line.strip().split(' ')
    2652     fp.close()
    2653     metadata = evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp,True)
    2654     title = "Results: ("+n+")"
    2655     t = ['Files: '+parFile,lblFile,' ']
    2656     n = ["Named parameters:"]
    2657     l = ['',"Labeled columns:"]
    2658     for key in sorted(metadata):
    2659         if key == "filename" or key.startswith('label_prm'): continue
    2660         if key in keyCols:
    2661             n += ["  {} = {}".format(key,metadata[key])]
    2662         elif key in lbldict.values():
    2663             l += ["  {} = {}".format(key,metadata[key])]
    2664         else:
    2665             t += ["** Unexpected:  {}".format(key,metadata[key])]
    2666     if type(metadata['filename']) is str:
    2667         l += ["","Filename: "+ metadata['filename']]
    2668     else:
    2669         l += ["","Filename(s): "]
    2670         for i,j in enumerate(metadata['filename']):
    2671             if i: l[-1] += ', '
    2672             l[-1] += j
    2673     t += n + l + ['','Unused columns:']
    2674     usedCols = list(lbldict.keys())
    2675     for i in keyCols.values(): usedCols += i
    2676     for i in range(len(items)):
    2677         if i in usedCols: continue
    2678         t += ["  {}: {}".format(i,items[i])]
    2679     dlg = G2G.G2SingleChoiceDialog(None,title,'Column metadata parse results',t,
    2680                                    monoFont=True,filterBox=False,size=(400,600),
    2681                                    style=wx.DEFAULT_DIALOG_STYLE|wx.RESIZE_BORDER|wx.CENTRE|wx.OK)
    2682     dlg.ShowModal()
    2683 
    26842242if __name__ == '__main__':
    26852243    import GSASIIdataGUI

  • trunk/GSASIIdataGUI.py

    r3807 r3814  
    815815                                rd.readfilename = rd.sumfile
    816816                            # Load generic metadata, as configured
    817                             G2IO.GetColumnMetadata(rd)
     817                            G2fil.GetColumnMetadata(rd)
    818818                            G2IO.LoadImage2Tree(rd.readfilename,self,rd.Comments,rd.Data,rd.Npix,rd.Image)
    819819                            rd_list.append(True) # save a stub the result before it is written over
    … …  
    30763076    def OnColMetaTest(self,event):
    30773077        'Test the .par/.*lbls pair for contents'
    3078         G2IO.testColumnMetadata(self)
     3078        G2imG.testColumnMetadata(self)
    30793079               
    30803080    def OnPowderFPA(self,event):

  • trunk/GSASIIfiles.py

    r3216 r3814  
    423423                fp.close()
    424424    return exporterlist
     425
     426def readColMetadata(imagefile):
     427    '''Reads image metadata from a column-oriented metadata table
     428    (1-ID style .par file). Called by :func:`GetColumnMetadata`
     429   
     430    The .par file has any number of columns separated by spaces.
     431    The directory for the file must be specified in
     432    Config variable ``Column_Metadata_directory``.
     433    As an index to the .par file a second "label file" must be specified with the
     434    same file root name as the .par file but the extension must be .XXX_lbls (where
     435    .XXX is the extension of the image) or if that is not present extension
     436    .lbls.
     437
     438    :param str imagefile: the full name of the image file (with extension, directory optional)
     439
     440    :returns: a dict with parameter values. Named parameters will have the type based on
     441       the specified Python function, named columns will be character strings
     442   
     443    The contents of the label file will look like this::
     444   
     445        # define keywords
     446        filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34
     447        distance: float | 75
     448        wavelength:lambda keV: 12.398425/float(keV)|9
     449        pixelSize:lambda x: [74.8, 74.8]|0
     450        ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4
     451        Temperature: float|53
     452        FreePrm2: int | 34 | Free Parm2 Label
     453        # define other variables
     454        0:day
     455        1:month
     456        2:date
     457        3:time
     458        4:year
     459        7:I_ring
     460
     461    This file contains three types of lines in any order.
     462     * Named parameters are evaluated with user-supplied Python code (see
     463       subsequent information). Specific named parameters are used
     464       to determine values that are used for image interpretation (see table,
     465       below). Any others are copied to the Comments subsection of the Image
     466       tree item.
     467     * Column labels are defined with a column number (integer) followed by
     468       a colon (:) and a label to be assigned to that column. All labeled
     469       columns are copied to the Image's Comments subsection.
     470     * Comments are any line that does not contain a colon.
     471
     472    Note that columns are numbered starting at zero.
     473
     474    Any named parameter may be defined provided it is not a valid integer,
     475    but the named parameters in the table have special meanings, as descibed.
     476    The parameter name is followed by a colon. After the colon, specify
     477    Python code that defines or specifies a function that will be called to
     478    generate a value for that parameter.
     479
     480    Note that several keywords, if defined in the Comments, will be found and
     481    placed in the appropriate section of the powder histogram(s)'s Sample
     482    Parameters after an integration: ``Temperature``,``Pressure``,``Time``,
     483    ``FreePrm1``,``FreePrm2``,``FreePrm3``,``Omega``,``Chi``, and ``Phi``.
     484
     485    After the Python code, supply a vertical bar (|) and then a list of one
     486    more more columns that will be supplied as arguments to that function.
     487
     488    Note that the labels for the three FreePrm items can be changed by
     489    including that label as a third item with an additional vertical bar. Labels
     490    will be ignored for any other named parameters.
     491   
     492    The examples above are discussed here:
     493
     494    ``filename:lambda x,y: "{}_{:0>6}".format(x,y)|33,34``
     495        Here the function to be used is defined with a lambda statement::
     496       
     497          lambda x,y: "{}_{:0>6}".format(x,y)
     498
     499        This function will use the format function to create a file name from the
     500        contents of columns 33 and 34. The first parameter (x, col. 33) is inserted directly into
     501        the file name, followed by a underscore (_), followed by the second parameter (y, col. 34),
     502        which will be left-padded with zeros to six characters (format directive ``:0>6``).
     503
     504        When there will be more than one image generated per line in the .par file, an alternate way to
     505        generate list of file names takes into account the number of images generated::
     506
     507          lambda x,y,z: ["{}_{:0>6}".format(x,int(y)+i) for i in range(int(z))]
     508
     509        Here a third parameter is used to specify the number of images generated, where
     510        the image number is incremented for each image.
     511         
     512    ``distance: float | 75``
     513        Here the contents of column 75 will be converted to a floating point number
     514        by calling float on it. Note that the spaces here are ignored.
     515       
     516    ``wavelength:lambda keV: 12.398425/float(keV)|9``
     517        Here we define an algebraic expression to convert an energy in keV to a
     518        wavelength and pass the contents of column 9 as that input energy
     519       
     520    ``pixelSize:lambda x: [74.8, 74.8]|0``
     521        In this case the pixel size is a constant (a list of two numbers). The first
     522        column is passed as an argument as at least one argument is required, but that
     523        value is not used in the expression.
     524
     525    ``ISOlikeDate: lambda dow,m,d,t,y:"{}-{}-{}T{} ({})".format(y,m,d,t,dow)|0,1,2,3,4``
     526        This example defines a parameter that takes items in the first five columns
     527        and formats them in a different way. This parameter is not one of the pre-defined
     528        parameter names below. Some external code could be used to change the month string
     529        (argument ``m``) to a integer from 1 to 12.
     530       
     531    ``FreePrm2: int | 34 | Free Parm2 Label``
     532        In this example, the contents of column 34 will be converted to an integer and
     533        placed as the second free-named parameter in the Sample Parameters after an
     534        integration. The label for this parameter will be changed to "Free Parm2 Label".
     535           
     536    **Pre-defined parameter names**
     537   
     538    =============  =========  ========  =====================================================
     539     keyword       required    type      Description
     540    =============  =========  ========  =====================================================
     541       filename    yes         str or   generates the file name prefix for the matching image
     542                               list     file (MyImage001 for file /tmp/MyImage001.tif) or
     543                                        a list of file names.
     544     polarization  no         float     generates the polarization expected based on the
     545                                        monochromator angle, defaults to 0.99.
     546       center      no         list of   generates the approximate beam center on the detector
     547                              2 floats  in mm, such as [204.8, 204.8].
     548       distance    yes        float     generates the distance from the sample to the detector
     549                                        in mm
     550       pixelSize   no         list of   generates the size of the pixels in microns such as
     551                              2 floats  [200.0, 200.0].
     552       wavelength  yes        float     generates the wavelength in Angstroms
     553    =============  =========  ========  =====================================================
     554   
     555    '''
     556    dir,fil = os.path.split(os.path.abspath(imagefile))
     557    imageName,ext = os.path.splitext(fil)
     558    if not GSASIIpath.GetConfigValue('Column_Metadata_directory'): return
     559    parfiles = glob.glob(os.path.join(GSASIIpath.GetConfigValue('Column_Metadata_directory'),'*.par'))
     560    if len(parfiles) == 0:
     561        print('Sorry, No Column metadata (.par) file found in '+
     562              GSASIIpath.GetConfigValue('Column_Metadata_directory'))
     563        return {}
     564    for parFil in parfiles: # loop over all .par files (hope just 1) in image dir until image is found
     565        parRoot = os.path.splitext(parFil)[0]
     566        for e in (ext+'_lbls','.lbls'):
     567            if os.path.exists(parRoot+e):
     568                lblFil = parRoot+e
     569                break
     570        else:
     571            print('Warning: No labels definitions found for '+parFil)
     572            continue
     573        labels,lbldict,keyCols,keyExp,errors = readColMetadataLabels(lblFil)
     574        if errors:
     575            print('Errors in labels file '+lblFil)
     576            for i in errors: print('  '+i)
     577            continue
     578        else:
     579            print('Read '+lblFil)
     580        # scan through each line in this .par file, looking for the matching image rootname
     581        fp = open(parFil,'Ur')
     582        for iline,line in enumerate(fp):
     583            items = line.strip().split(' ')
     584            nameList = keyExp['filename'](*[items[j] for j in keyCols['filename']])
     585            if type(nameList) is str:
     586                if nameList != imageName: continue
     587                name = nameList
     588            else:
     589                for name in nameList:
     590                    if name == imageName: break # got a match
     591                else:
     592                    continue
     593            # parse the line and finish
     594            metadata = evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp)
     595            metadata['par file'] = parFil
     596            metadata['lbls file'] = lblFil
     597            print("Metadata read from {} line {}".format(parFil,iline+1))
     598            fp.close()
     599            return metadata
     600        else:
     601            print("Image {} not found in {}".format(imageName,parFil))
     602            fp.close()
     603            continue
     604        fp.close()
     605    else:
     606        print("Warning: No .par metadata for image {}".format(imageName))
     607        return {}
     608
     609def readColMetadataLabels(lblFil):
     610    '''Read the .*lbls file and setup for metadata assignments
     611    '''
     612    lbldict = {}
     613    keyExp = {}
     614    keyCols = {}
     615    labels = {}
     616    errors = []
     617    fp = open(lblFil,'Ur')         # read column labels
     618    for iline,line in enumerate(fp): # read label definitions
     619        line = line.strip()
     620        if not line or line[0] == '#': continue # comments
     621        items = line.split(':')
     622        if len(items) < 2: continue # lines with no colon are also comments
     623        # does this line a definition for a named parameter?
     624        key = items[0]
     625        try:
     626            int(key)
     627        except ValueError: # try as named parameter since not a valid number
     628            items = line.split(':',1)[1].split('|')
     629            try:
     630                f = eval(items[0]) # compile the expression
     631                if not callable(f):
     632                    errors += ['Expression "{}" for key {} is not a function (line {})'.
     633                           format(items[0],key,iline)]
     634                    continue
     635                keyExp[key] = f
     636            except Exception as msg:
     637                errors += ['Expression "{}" for key {} is not valid (line {})'.
     638                           format(items[0],key,iline)]
     639                errors += [str(msg)]
     640                continue
     641            keyCols[key] = [int(i) for i in items[1].strip().split(',')]
     642            if key.lower().startswith('freeprm') and len(items) > 2:
     643                labels[key] = items[2]
     644            continue
     645        if len(items) == 2: # simple column definition
     646            lbldict[int(items[0])] = items[1]
     647    fp.close()
     648    if 'filename' not in keyExp:
     649        errors += ["File {} is invalid. No valid filename expression.".format(lblFil)]
     650    return labels,lbldict,keyCols,keyExp,errors
     651
     652def evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp,ShowError=False):
     653    '''Evaluate the metadata for a line in the .par file
     654    '''
     655    metadata = {lbldict[j]:items[j] for j in lbldict}
     656    named = {}
     657    for key in keyExp:
     658        try:
     659            res = keyExp[key](*[items[j] for j in keyCols[key]])
     660        except:
     661            if ShowError:
     662                res = "*** error ***"
     663            else:
     664                continue
     665        named[key] = res
     666    metadata.update(named)
     667    for lbl in labels: # add labels for FreePrm's
     668        metadata['label_'+lbl[4:].lower()] = labels[lbl]
     669    return metadata
     670
     671def GetColumnMetadata(reader):
     672    '''Add metadata to an image from a column-type metadata file
     673    using :func:`readColMetadata`
     674   
     675    :param reader: a reader object from reading an image
     676   
     677    '''
     678    if not GSASIIpath.GetConfigValue('Column_Metadata_directory'): return
     679    parParms = readColMetadata(reader.readfilename)
     680    if not parParms: return # check for read failure
     681    specialKeys = ('filename',"polarization", "center", "distance", "pixelSize", "wavelength",)
     682    reader.Comments = ['Metadata from {} assigned by {}'.format(parParms['par file'],parParms['lbls file'])]
     683    for key in parParms:
     684        if key in specialKeys+('par file','lbls file'): continue
     685        reader.Comments += ["{} = {}".format(key,parParms[key])]
     686    if "polarization" in parParms:
     687        reader.Data['PolaVal'][0] = parParms["polarization"]
     688    else:
     689        reader.Data['PolaVal'][0] = 0.99
     690    if "center" in parParms:
     691        reader.Data['center'] = parParms["center"]
     692    if "pixelSize" in parParms:
     693        reader.Data['pixelSize'] = parParms["pixelSize"]
     694    if "wavelength" in parParms:
     695        reader.Data['wavelength'] = parParms['wavelength']
     696    else:
     697        print('Error: wavelength not defined in {}'.format(parParms['lbls file']))
     698    if "distance" in parParms:
     699        reader.Data['distance'] = parParms['distance']
     700        reader.Data['setdist'] = parParms['distance']
     701    else:
     702        print('Error: distance not defined in {}'.format(parParms['lbls file']))
     703
     704def LoadControls(Slines,data):
     705    'Read values from a .imctrl (Image Controls) file'
     706    cntlList = ['color','wavelength','distance','tilt','invert_x','invert_y','type','Oblique',
     707        'fullIntegrate','outChannels','outAzimuths','LRazimuth','IOtth','azmthOff','DetDepth',
     708        'calibskip','pixLimit','cutoff','calibdmin','Flat Bkg','varyList','setdist',
     709        'PolaVal','SampleAbs','dark image','background image','twoth']
     710    save = {}
     711    for S in Slines:
     712        if S[0] == '#':
     713            continue
     714        [key,val] = S.strip().split(':',1)
     715        if key in ['type','calibrant','binType','SampleShape','color',]:    #strings
     716            save[key] = val
     717        elif key in ['varyList',]:
     718            save[key] = eval(val)   #dictionary
     719        elif key in ['rotation']:
     720            save[key] = float(val)
     721        elif key in ['center',]:
     722            if ',' in val:
     723                save[key] = eval(val)
     724            else:
     725                vals = val.strip('[] ').split()
     726                save[key] = [float(vals[0]),float(vals[1])]
     727        elif key in cntlList:
     728            save[key] = eval(val)
     729    data.update(save)
     730
     731def WriteControls(filename,data):
     732    'Write current values to a .imctrl (Image Controls) file'
     733    File = open(filename,'w')
     734    keys = ['type','color','wavelength','calibrant','distance','center','Oblique',
     735            'tilt','rotation','azmthOff','fullIntegrate','LRazimuth','setdist',
     736            'IOtth','outChannels','outAzimuths','invert_x','invert_y','DetDepth',
     737            'calibskip','pixLimit','cutoff','calibdmin','Flat Bkg','varyList',
     738            'binType','SampleShape','PolaVal','SampleAbs','dark image','background image',
     739            'twoth']
     740    for key in keys:
     741        if key not in data:     #uncalibrated!
     742            continue
     743        File.write(key+':'+str(data[key])+'\n')
     744    File.close()
     745   

  • trunk/GSASIIimgGUI.py

    r3779 r3814  
    3636import GSASIIplot as G2plt
    3737import GSASIIIO as G2IO
     38import GSASIIfiles as G2fil
    3839import GSASIIdataGUI as G2gd
    3940import GSASIIctrlGUI as G2G
    … …  
    410411                # make sure extension is .imctrl
    411412                filename = os.path.splitext(filename)[0]+'.imctrl'
    412                 WriteControls(filename,data)
     413                G2fil.WriteControls(filename,data)
    413414        finally:
    414415            dlg.Destroy()
    415 
    416     def WriteControls(filename,data):
    417         File = open(filename,'w')
    418         keys = ['type','color','wavelength','calibrant','distance','center','Oblique',
    419             'tilt','rotation','azmthOff','fullIntegrate','LRazimuth','setdist',
    420             'IOtth','outChannels','outAzimuths','invert_x','invert_y','DetDepth',
    421             'calibskip','pixLimit','cutoff','calibdmin','Flat Bkg','varyList',
    422             'binType','SampleShape','PolaVal','SampleAbs','dark image','background image',
    423             'twoth']
    424         for key in keys:
    425             if key not in data:     #uncalibrated!
    426                 continue
    427             File.write(key+':'+str(data[key])+'\n')
    428         File.close()
    429416       
    430417    def OnSaveMultiControls(event):
    … …  
    473460                                    + '.imctrl')
    474461            print('writing '+filename)
    475             WriteControls(filename,data)
    476            
    477     def LoadControls(Slines,data):
    478         cntlList = ['color','wavelength','distance','tilt','invert_x','invert_y','type','Oblique',
    479             'fullIntegrate','outChannels','outAzimuths','LRazimuth','IOtth','azmthOff','DetDepth',
    480             'calibskip','pixLimit','cutoff','calibdmin','Flat Bkg','varyList','setdist',
    481             'PolaVal','SampleAbs','dark image','background image','twoth']
    482         save = {}
    483         for S in Slines:
    484             if S[0] == '#':
    485                 continue
    486             [key,val] = S.strip().split(':',1)
    487             if key in ['type','calibrant','binType','SampleShape','color',]:    #strings
    488                 save[key] = val
    489             elif key in ['varyList',]:
    490                 save[key] = eval(val)   #dictionary
    491             elif key in ['rotation']:
    492                 save[key] = float(val)
    493             elif key in ['center',]:
    494                 if ',' in val:
    495                     save[key] = eval(val)
    496                 else:
    497                     vals = val.strip('[] ').split()
    498                     save[key] = [float(vals[0]),float(vals[1])]
    499             elif key in cntlList:
    500                 save[key] = eval(val)
    501         data.update(save)
    502         # next line removed. Previous updates tree contents. The next
    503         # makes a copy of data, puts it into tree and "disconnects" data
    504         # from tree contents (later changes to data are lost!)
    505         #G2frame.GPXtree.SetItemPyData(G2gd.GetGPXtreeItemId(G2frame,G2frame.Image, 'Image Controls'),copy.deepcopy(data))
    506        
     462            G2fil.WriteControls(filename,data)
    507463           
    508464    def OnLoadControls(event):
    … …  
    517473                Slines = File.readlines()
    518474                File.close()
    519                 LoadControls(Slines,data)
     475                G2fil.LoadControls(Slines,data)
    520476        finally:
    521477            dlg.Destroy()
    … …  
    562518                    imctrls = G2frame.GPXtree.GetItemPyData(G2gd.GetGPXtreeItemId(G2frame,Id,'Image Controls'))
    563519                    Slines = controlsDict[imctrls['twoth']]
    564                     LoadControls(Slines,imctrls)
     520                    G2fil.LoadControls(Slines,imctrls)
    565521        finally:
    566522            dlg.Destroy()
    … …  
    35503506# Autointegration end
    35513507###########################################################################
     3508
     3509def testColumnMetadata(G2frame):
     3510    '''Test the column-oriented metadata parsing, as implemented at 1-ID, by showing results
     3511    when using a .par and .lbls pair.
     3512   
     3513     * Select a .par file, if more than one in selected dir.
     3514     * Select the .*lbls file, if more than one matching .par file.
     3515     * Parse the .lbls file, showing errors if encountered; loop until errors are fixed.
     3516     * Search for an image or a line in the .par file and show the results when interpreted
     3517     
     3518    See :func:`GSASIIfiles.readColMetadata` for more details.
     3519    '''
     3520    if not GSASIIpath.GetConfigValue('Column_Metadata_directory'):
     3521        G2G.G2MessageBox(G2frame,'The configuration option for I-ID Metadata is not set.\n'+
     3522                         'Please use the File/Preferences menu to set Column_Metadata_directory',
     3523                         'Warning')
     3524        return
     3525    parFiles = glob.glob(os.path.join(GSASIIpath.GetConfigValue('Column_Metadata_directory'),'*.par'))
     3526    if not parFiles:
     3527        G2G.G2MessageBox(G2frame,'No .par files found in directory {}. '
     3528                         .format(GSASIIpath.GetConfigValue('Column_Metadata_directory'))+
     3529                         '\nThis is set by config variable Column_Metadata_directory '+
     3530                         '(Set in File/Preferences menu).',
     3531                         'Warning')
     3532        return
     3533    parList = []
     3534    for parFile in parFiles:
     3535        lblList = []
     3536        parRoot = os.path.splitext(parFile)[0]
     3537        for f in glob.glob(parRoot+'.*lbls'):
     3538            if os.path.exists(f): lblList.append(f)
     3539        if not len(lblList):
     3540            continue
     3541        parList.append(parFile)
     3542    if len(parList) == 0:
     3543        G2G.G2MessageBox(G2frame,'No .lbls or .EXT_lbls file found for .par file(s) in directory {}. '
     3544                         .format(GSASIIpath.GetConfigValue('Column_Metadata_directory'))+
     3545                         '\nThis is set by config variable Column_Metadata_directory '+
     3546                         '(Set in File/Preferences menu).',
     3547                         'Warning')
     3548        return
     3549    elif len(parList) == 1:
     3550        parFile = parList[0]
     3551    else:
     3552        dlg = G2G.G2SingleChoiceDialog(G2frame,
     3553                'More than 1 .par file found. (Better if only 1!). Choose the one to test in '+
     3554                GSASIIpath.GetConfigValue('Column_Metadata_directory'),
     3555                'Choose .par file', [os.path.split(i)[1] for i in parList])
     3556        if dlg.ShowModal() == wx.ID_OK:
     3557            parFile = parList[dlg.GetSelection()]
     3558            dlg.Destroy()
     3559        else:
     3560            dlg.Destroy()
     3561            return
     3562    # got .par file; now work on .*lbls file
     3563    lblList = []
     3564    parRoot = os.path.splitext(parFile)[0]
     3565    for f in glob.glob(parRoot+'.*lbls'):
     3566        if os.path.exists(f): lblList.append(f)
     3567    if not len(lblList):
     3568        raise Exception('How did this happen! No .*lbls files for '+parFile)
     3569    elif len(lblList) == 1:
     3570        lblFile = lblList[0]
     3571    else:
     3572        dlg = G2G.G2SingleChoiceDialog(G2frame,
     3573                'Select label file for .par file '+parFile,
     3574                'Choose label file', [os.path.split(i)[1] for i in lblList])
     3575        if dlg.ShowModal() == wx.ID_OK:
     3576            lblFile = lblList[dlg.GetSelection()]
     3577            dlg.Destroy()
     3578        else:
     3579            dlg.Destroy()
     3580            return
     3581    # parse the labels file
     3582    errors = True
     3583    while errors:
     3584        labels,lbldict,keyCols,keyExp,errors = G2fil.readColMetadataLabels(lblFile)
     3585        if errors:
     3586            t = "Error reading file "+lblFile
     3587            for l in errors:
     3588                t += '\n'
     3589                t += l
     3590            t += "\n\nPlease edit the file and press OK (or Cancel to quit)"
     3591            dlg = wx.MessageDialog(G2frame,message=t,
     3592                caption="Read Error",style=wx.ICON_ERROR| wx.OK|wx.STAY_ON_TOP|wx.CANCEL)
     3593            if dlg.ShowModal() != wx.ID_OK: return           
     3594    # request a line number, read that line
     3595    dlg = G2G.SingleStringDialog(G2frame,'Read what',
     3596                                 'Enter a line number or an image file name (-1=last line)',
     3597                                 '-1',size=(400,-1))
     3598    if dlg.Show():
     3599        fileorline = dlg.GetValue()
     3600        dlg.Destroy()
     3601    else:
     3602        dlg.Destroy()
     3603        return
     3604    # and report the generated key pairs in metadata dict
     3605    linenum = None
     3606    try:
     3607        linenum = int(fileorline)
     3608    except:
     3609        imageName = os.path.splitext(os.path.split(fileorline)[1])[0]
     3610
     3611    fp = open(parFile,'Ur')
     3612    for iline,line in enumerate(fp):
     3613        if linenum is not None:
     3614            if iline == linenum:
     3615                items = line.strip().split(' ')
     3616                n = "Line {}".format(iline)
     3617                break
     3618            else:
     3619                continue
     3620        else:
     3621            items = line.strip().split(' ')
     3622            nameList = keyExp['filename'](*[items[j] for j in keyCols['filename']])
     3623            if type(nameList) is str:
     3624                if nameList != imageName: continue
     3625                name = nameList
     3626                break
     3627            else:
     3628                for name in nameList:
     3629                    print (name,name == imageName)
     3630                    if name == imageName:
     3631                        n = "Image {} found in line {}".format(imageName,iline)
     3632                        break # got a match
     3633                else:
     3634                    continue
     3635                break
     3636    else:
     3637        if linenum is not None:
     3638            n = "Line {}".format(iline)
     3639        else:
     3640            n = "Image {} not found. Reporting line {}".format(imageName,iline)
     3641        items = line.strip().split(' ')
     3642    fp.close()
     3643    metadata = G2fil.evalColMetadataDicts(items,labels,lbldict,keyCols,keyExp,True)
     3644    title = "Results: ("+n+")"
     3645    t = ['Files: '+parFile,lblFile,' ']
     3646    n = ["Named parameters:"]
     3647    l = ['',"Labeled columns:"]
     3648    for key in sorted(metadata):
     3649        if key == "filename" or key.startswith('label_prm'): continue
     3650        if key in keyCols:
     3651            n += ["  {} = {}".format(key,metadata[key])]
     3652        elif key in lbldict.values():
     3653            l += ["  {} = {}".format(key,metadata[key])]
     3654        else:
     3655            t += ["** Unexpected:  {}".format(key,metadata[key])]
     3656    if type(metadata['filename']) is str:
     3657        l += ["","Filename: "+ metadata['filename']]
     3658    else:
     3659        l += ["","Filename(s): "]
     3660        for i,j in enumerate(metadata['filename']):
     3661            if i: l[-1] += ', '
     3662            l[-1] += j
     3663    t += n + l + ['','Unused columns:']
     3664    usedCols = list(lbldict.keys())
     3665    for i in keyCols.values(): usedCols += i
     3666    for i in range(len(items)):
     3667        if i in usedCols: continue
     3668        t += ["  {}: {}".format(i,items[i])]
     3669    dlg = G2G.G2SingleChoiceDialog(None,title,'Column metadata parse results',t,
     3670                                   monoFont=True,filterBox=False,size=(400,600),
     3671                                   style=wx.DEFAULT_DIALOG_STYLE|wx.RESIZE_BORDER|wx.CENTRE|wx.OK)
     3672    dlg.ShowModal()

  • trunk/GSASIIscriptable.py

    r3811 r3814  
    571571import GSASIIpwd as G2pwd
    572572import GSASIIstrMain as G2strMain
     573#import GSASIIIO as G2IO
    573574import GSASIIstrIO as G2strIO
    574575import GSASIIspc as G2spc
    575576import GSASIIElem as G2elem
    576 
    577 
    578 # Delay imports to not slow down small scripts
    579 G2fil = None
    580 PwdrDataReaders = []
    581 PhaseReaders = []
     577import GSASIIfiles as G2fil
     578
     579# Delay imports to not slow down small scripts that don't need them
     580Readers = {'Pwdr':[], 'Phase':[], 'Image':[]}
     581'''Readers by reader type'''
    582582exportersByExtension = {}
    583583'''Specifies the list of extensions that are supported for Powder data export'''
    … …  
    585585def LoadG2fil():
    586586    """Delay importing this module, it is slow"""
    587     global G2fil
    588     if G2fil is None:
    589         import GSASIIfiles
    590         G2fil = GSASIIfiles
    591         global PwdrDataReaders
    592         global PhaseReaders
    593         PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
    594         PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
    595         AllExporters = G2fil.LoadExportRoutines(None)
    596         global exportersByExtension
    597         exportersByExtension = {}
    598         for obj in AllExporters:
    599             try:
    600                 obj.Writer
    601             except AttributeError:
    602                 continue
    603             for typ in obj.exporttype:
    604                 if typ not in exportersByExtension:
    605                     exportersByExtension[typ] = {obj.extension:obj}
    606                 else:
    607                     exportersByExtension[typ][obj.extension] = obj
     587    if len(Readers['Pwdr']) > 0: return
     588    # initialize imports
     589    Readers['Pwdr'] = G2fil.LoadImportRoutines("pwd", "Powder_Data")
     590    Readers['Phase'] = G2fil.LoadImportRoutines("phase", "Phase")
     591    Readers['Image'] = G2fil.LoadImportRoutines("img", "Image")
     592
     593    # initialize exports
     594    for obj in exportersByExtension:
     595        try:
     596            obj.Writer
     597        except AttributeError:
     598            continue
     599        for typ in obj.exporttype:
     600            if typ not in exportersByExtension:
     601                exportersByExtension[typ] = {obj.extension:obj}
     602            else:
     603                exportersByExtension[typ][obj.extension] = obj
    608604
    609605def LoadDictFromProjFile(ProjFile):
    … …  
    11981194
    11991195
    1200 class G2Project(G2ObjectWrapper):
    1201     """
    1202     Represents an entire GSAS-II project.
     1196class G2Project(G2ObjectWrapper):   
     1197    """Represents an entire GSAS-II project.
    12031198
    12041199    :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
    … …  
    13351330        datafile = os.path.abspath(os.path.expanduser(datafile))
    13361331        iparams = os.path.abspath(os.path.expanduser(iparams))
    1337         pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint,bank=databank)
     1332        pwdrreaders = import_generic(datafile, Readers['Pwdr'],fmthint=fmthint,bank=databank)
    13381333        histname, new_names, pwdrdata = load_pwd_from_reader(
    13391334                                          pwdrreaders[0], iparams,
    … …  
    14451440        :param str phasename: The name of the new phase, or None for the default
    14461441        :param list histograms: The names of the histograms to associate with
    1447             this phase. Use proj.Histograms() to add to all histograms.
     1442            this phase. Use proj.histograms() to add to all histograms.
    14481443        :param str fmthint: If specified, only importers where the format name
    14491444          (reader.formatName, as shown in Import menu) contains the
    … …  
    14601455
    14611456        # TODO handle multiple phases in a file
    1462         phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
     1457        phasereaders = import_generic(phasefile, Readers['Phase'], fmthint=fmthint)
    14631458        phasereader = phasereaders[0]
    14641459       
    … …  
    15771572            :meth:`G2Project.phase`
    15781573            :meth:`G2Project.phases`
    1579             """
     1574        """
    15801575        if isinstance(histname, G2PwdrData):
    15811576            if histname.proj == self:
    … …  
    15931588                return histogram
    15941589
    1595     def histograms(self):
    1596         """Return a list of all histograms, as
    1597         :class:`G2PwdrData` objects
     1590    def histograms(self, typ=None):
     1591        """Return a list of all histograms, as :class:`G2PwdrData` objects
     1592
     1593        For now this only finds Powder/Single Xtal histograms, since that is all that is
     1594        currently implemented in this module.
     1595
     1596        :param ste typ: The prefix (type) the histogram such as 'PWDR '. If None
     1597          (the default) all known histograms types are found.
     1598        :returns: a list of objects
    15981599
    15991600        .. seealso::
    1600             :meth:`G2Project.histograms`
     1601            :meth:`G2Project.histogram`
    16011602            :meth:`G2Project.phase`
    16021603            :meth:`G2Project.phases`
    1603             """
     1604        """
    16041605        output = []
    16051606        # loop through each tree entry. If it is more than one level (more than one item in the
    1606         # list of names) then it must be a histogram, unless labeled Phases or Restraints
    1607         for obj in self.names:
    1608             if len(obj) > 1 and obj[0] != u'Phases' and obj[0] != u'Restraints':
    1609                 output.append(self.histogram(obj[0]))
     1607        # list of names). then it must be a histogram, unless labeled Phases or Restraints
     1608        if typ is None:
     1609            for obj in self.names:
     1610                if obj[0].startswith('PWDR ') or obj[0].startswith('HKLF '):
     1611                    output.append(self.histogram(obj[0]))
     1612        else:
     1613            for obj in self.names:
     1614                if len(obj) > 1 and obj[0].startswith(typ):
     1615                    output.append(self.histogram(obj[0]))
    16101616        return output
    16111617
    … …  
    16581664        return []
    16591665
     1666    def _images(self):
     1667        """Returns a list of all the phases in the project.
     1668        """
     1669        return [i[0] for i in self.names if i[0].startswith('IMG ')]
     1670   
     1671    def image(self, imageRef):
     1672        """
     1673        Gives an object representing the specified image in this project.
     1674
     1675        :param str imageRef: A reference to the desired image. Either the Image
     1676          tree name (str), the image's index (int) or
     1677          a image object (:class:`G2Image`)
     1678        :returns: A :class:`G2Image` object
     1679        :raises: KeyError
     1680
     1681        .. seealso::
     1682            :meth:`G2Project.images`
     1683            """
     1684        if isinstance(imageRef, G2Image):
     1685            if imageRef.proj == self:
     1686                return imageRef
     1687            else:
     1688                raise Exception("Image {} not in current selected project".format(imageRef.name))
     1689        if imageRef in self._images():
     1690            return G2Image(self.data[imageRef], imageRef, self)
     1691
     1692        try:
     1693            # imageRef should be an index
     1694            num = int(imageRef)
     1695            imageRef = self._images()[num]
     1696            return G2Image(self.data[imageRef], imageRef, self)
     1697        except ValueError:
     1698            raise Exception("imageRef {} not an object, name or image index in current selected project"
     1699                                .format(imageRef))
     1700        except IndexError:
     1701            raise Exception("imageRef {} out of range (max={}) in current selected project"
     1702                                .format(imageRef,len(self._images())-1))
     1703           
     1704    def images(self):
     1705        """
     1706        Returns a list of all the images in the project.
     1707
     1708        :returns: A list of :class:`G2Image` objects
     1709        """
     1710        return [G2Image(self.data[i],i,self) for i in self._images()]
     1711   
    16601712    def update_ids(self):
    16611713        """Makes sure all phases and histograms have proper hId and pId"""
    … …  
    19522004        return G2obj.G2VarObj(phase, hist, varname, atomId)
    19532005
     2006    def add_image(self, imagefile, fmthint=None, defaultImage=None):
     2007        """Load an image into a project
     2008
     2009        :param str imagefile: The image file to read, a filename.
     2010        :param str fmthint: If specified, only importers where the format name
     2011          (reader.formatName, as shown in Import menu) contains the
     2012          supplied string will be tried as importers. If not specified, all
     2013          importers consistent with the file extension will be tried
     2014          (equivalent to "guess format" in menu).
     2015        :param str defaultImage: The name of an image to use as a default for
     2016          setting parameters for the image file to read.
     2017
     2018        :returns: a list of G2Image object for the added image(s) [this routine
     2019          has not yet been tested with files with multiple images...]
     2020        """
     2021        LoadG2fil()
     2022        imagefile = os.path.abspath(os.path.expanduser(imagefile))
     2023        readers = import_generic(imagefile, Readers['Image'], fmthint=fmthint)
     2024        objlist = []
     2025        for rd in readers:
     2026            if rd.SciPy:        #was default read by scipy; needs 1 time fixes
     2027                print('TODO: Image {} read by SciPy. Parameters likely need editing'.format(imagefile))
     2028                #see this: G2IO.EditImageParms(self,rd.Data,rd.Comments,rd.Image,imagefile)
     2029                rd.SciPy = False
     2030            rd.readfilename = imagefile
     2031            if rd.repeatcount == 1 and not rd.repeat: # skip image number if only one in set
     2032                rd.Data['ImageTag'] = None
     2033            else:
     2034                rd.Data['ImageTag'] = rd.repeatcount
     2035            rd.Data['formatName'] = rd.formatName
     2036            if rd.sumfile:
     2037                rd.readfilename = rd.sumfile
     2038            # Load generic metadata, as configured
     2039            G2fil.GetColumnMetadata(rd)
     2040            # Code from G2IO.LoadImage2Tree(rd.readfilename,self,rd.Comments,rd.Data,rd.Npix,rd.Image)
     2041            Imax = np.amax(rd.Image)
     2042            ImgNames = [i[0] for i in self.names if i[0].startswith('IMG ')]
     2043            TreeLbl = 'IMG '+os.path.basename(imagefile)
     2044            ImageTag = rd.Data.get('ImageTag')
     2045            if ImageTag:
     2046                TreeLbl += ' #'+'%04d'%(ImageTag)
     2047                imageInfo = (imagefile,ImageTag)
     2048            else:
     2049                imageInfo = imagefile
     2050            TreeName = G2obj.MakeUniqueLabel(TreeLbl,ImgNames)
     2051            # MT dict to contain image info
     2052            ImgDict = {}
     2053            ImgDict['data'] = [rd.Npix,imageInfo]
     2054            ImgDict['Comments'] = rd.Comments
     2055            if defaultImage:
     2056                if isinstance(defaultImage, G2Image):
     2057                    if defaultImage.proj == self:
     2058                        defaultImage = self.data[defaultImage.name]['data']
     2059                    else:
     2060                        raise Exception("Image {} not in current selected project".format(defaultImage.name))
     2061                elif defaultImage in self._images():
     2062                    defaultImage = self.data[defaultImage]['data']
     2063                else:
     2064                    defaultImage = None
     2065            Data = rd.Data
     2066            if defaultImage:
     2067                Data = copy.copy(defaultImage)
     2068                Data['showLines'] = True
     2069                Data['ring'] = []
     2070                Data['rings'] = []
     2071                Data['cutoff'] = 10.
     2072                Data['pixLimit'] = 20
     2073                Data['edgemin'] = 100000000
     2074                Data['calibdmin'] = 0.5
     2075                Data['calibskip'] = 0
     2076                Data['ellipses'] = []
     2077                Data['calibrant'] = ''
     2078                Data['GonioAngles'] = [0.,0.,0.]
     2079                Data['DetDepthRef'] = False
     2080            else:
     2081                Data['type'] = 'PWDR'
     2082                Data['color'] = GSASIIpath.GetConfigValue('Contour_color','Paired')
     2083                if 'tilt' not in Data:          #defaults if not preset in e.g. Bruker importer
     2084                    Data['tilt'] = 0.0
     2085                    Data['rotation'] = 0.0
     2086                    Data['pixLimit'] = 20
     2087                    Data['calibdmin'] = 0.5
     2088                    Data['cutoff'] = 10.
     2089                Data['showLines'] = False
     2090                Data['calibskip'] = 0
     2091                Data['ring'] = []
     2092                Data['rings'] = []
     2093                Data['edgemin'] = 100000000
     2094                Data['ellipses'] = []
     2095                Data['GonioAngles'] = [0.,0.,0.]
     2096                Data['DetDepth'] = 0.
     2097                Data['DetDepthRef'] = False
     2098                Data['calibrant'] = ''
     2099                Data['IOtth'] = [5.0,50.0]
     2100                Data['LRazimuth'] = [0.,180.]
     2101                Data['azmthOff'] = 0.0
     2102                Data['outChannels'] = 2250
     2103                Data['outAzimuths'] = 1
     2104                Data['centerAzm'] = False
     2105                Data['fullIntegrate'] = GSASIIpath.GetConfigValue('fullIntegrate',True)
     2106                Data['setRings'] = False
     2107                Data['background image'] = ['',-1.0]                           
     2108                Data['dark image'] = ['',-1.0]
     2109                Data['Flat Bkg'] = 0.0
     2110                Data['Oblique'] = [0.5,False]
     2111            Data['setDefault'] = False
     2112            Data['range'] = [(0,Imax),[0,Imax]]
     2113            ImgDict['Image Controls'] = Data
     2114            ImgDict['Masks'] = {'Points':[],'Rings':[],'Arcs':[],'Polygons':[],
     2115                                'Frames':[],'Thresholds':[(0,Imax),[0,Imax]]}
     2116            ImgDict['Stress/Strain']  = {'Type':'True','d-zero':[],'Sample phi':0.0,
     2117                                             'Sample z':0.0,'Sample load':0.0}
     2118            self.names.append([TreeName]+['Comments','Image Controls','Masks','Stress/Strain'])
     2119            self.data[TreeName] = ImgDict
     2120            del rd.Image
     2121            objlist.append(G2Image(self.data[TreeName], TreeName, self))
     2122        return objlist
    19542123
    19552124class G2AtomRecord(G2ObjectWrapper):
    … …  
    29413110                    print(u'Unknown HAP key: '+key)
    29423111
     3112class G2Image(G2ObjectWrapper):
     3113    """Wrapper for an IMG tree entry, containing an image and various metadata.
     3114    """
     3115    # parameters in
     3116    ControlList = {
     3117        'int': ['calibskip', 'pixLimit', 'edgemin', 'outChannels',
     3118                    'outAzimuths'],
     3119        'float': ['cutoff', 'setdist', 'wavelength', 'Flat Bkg',
     3120                      'azmthOff', 'tilt', 'calibdmin', 'rotation',
     3121                      'distance', 'DetDepth'],
     3122        'bool': ['setRings', 'setDefault', 'centerAzm', 'fullIntegrate',
     3123                     'DetDepthRef', 'showLines'],
     3124        'str': ['SampleShape', 'binType', 'formatName', 'color',
     3125                    'type', 'calibrant'],
     3126        'list': ['GonioAngles', 'IOtth', 'LRazimuth', 'Oblique', 'PolaVal',
     3127                   'SampleAbs', 'center', 'ellipses', 'linescan',
     3128                    'pixelSize', 'range', 'ring', 'rings', 'size', ],
     3129        'dict': ['varylist'],
     3130#        'image': ['background image', 'dark image', 'Gain map'],
     3131        }
     3132    '''Defines the items known to exist in the Image Controls tree section
     3133    and their data types.
     3134    '''
     3135
     3136    def __init__(self, data, name, proj):
     3137        self.data = data
     3138        self.name = name
     3139        self.proj = proj
     3140
     3141    def setControl(self,arg,value):
     3142        '''Set an Image Controls parameter in the current image.
     3143        If the parameter is not found an exception is raised.
     3144
     3145        :param str arg: the name of a parameter (dict entry) in the
     3146          image. The parameter must be found in :data:`ControlList`
     3147          or an exception is raised.
     3148        :param value: the value to set the parameter. The value is
     3149          cast as the appropriate type from :data:`ControlList`.
     3150        '''
     3151        for typ in self.ControlList:
     3152            if arg in self.ControlList[typ]: break
     3153        else:
     3154            raise Exception('arg {} not defined in G2Image.setControl'
     3155                                .format(arg))
     3156        try:
     3157            if typ == 'int':
     3158                self.data['Image Controls'][arg] = int(value)
     3159            elif typ == 'float':
     3160                self.data['Image Controls'][arg] = float(value)
     3161            elif typ == 'bool':
     3162                self.data['Image Controls'][arg] = bool(value)
     3163            elif typ == 'str':
     3164                self.data['Image Controls'][arg] = str(value)
     3165            elif typ == 'list':
     3166                self.data['Image Controls'][arg] = list(value)
     3167            elif typ == 'dict':
     3168                self.data['Image Controls'][arg] = dict(value)
     3169            else:
     3170                raise Exception('Unknown type {} for arg {} in  G2Image.setControl'
     3171                                    .format(typ,arg))
     3172        except:
     3173            raise Exception('Error formatting value {} as type {} for arg {} in  G2Image.setControl'
     3174                                    .format(value,typ,arg))
     3175
     3176    def getControl(self,arg):
     3177        '''Return an Image Controls parameter in the current image.
     3178        If the parameter is not found an exception is raised.
     3179
     3180        :param str arg: the name of a parameter (dict entry) in the
     3181          image.
     3182        :returns: the value as a int, float, list,...
     3183        '''
     3184        if arg in self.data['Image Controls']:
     3185            return self.data['Image Controls'][arg]
     3186        raise Exception('arg {} not defined in G2Image.getControl'.format(arg))
     3187
     3188    def findControl(self,arg):
     3189        '''Finds the Image Controls parameter(s) in the current image
     3190        that match the string in arg.
     3191
     3192          Example: findControl('calib') will return
     3193          [['calibskip', 'int'], ['calibdmin', 'float'], ['calibrant', 'str']]
     3194
     3195        :param str arg: a string containing part of the name of a
     3196          parameter (dict entry) in the image's Image Controls.
     3197        :returns: a list of matching entries in form
     3198          [['item','type'], ['item','type'],...] where each 'item' string
     3199          contains the sting in arg.
     3200        '''
     3201        matchList = []
     3202        for typ in self.ControlList:
     3203            for item in self.ControlList[typ]:
     3204                if arg in item:
     3205                    matchList.append([item,typ])
     3206        return matchList
     3207
     3208    def setControlFile(self,typ,imageRef,mult=None):
     3209        '''Set a image to be used as a background/dark/gain map image
     3210
     3211        :param str typ: specifies image type, which must be one of:
     3212           'background image', 'dark image', 'gain map'; N.B. only the first
     3213           four characters must be specified and case is ignored.
     3214        :param imageRef:
     3215        :param float mult:
     3216        '''
     3217#        'image': ['background image', 'dark image', 'Gain map'],
     3218        if 'back' in typ.lower():
     3219            key = 'background image'
     3220            mult = float(mult)
     3221        elif 'dark' in typ.lower():
     3222            key = 'dark image'
     3223            mult = float(mult)
     3224        elif 'gain' in typ.lower():
     3225            #key = 'Gain map'
     3226            if mult is not None:
     3227                print('Ignoring multiplier for Gain map')
     3228            mult = None
     3229        else:
     3230            raise Exception("Invalid typ {} for setControlFile".format(typ))
     3231        imgNam = self.proj.image(imageRef).name
     3232        if mult is None:
     3233            self.data['Image Controls']['Gain map'] = imgNam
     3234        else:
     3235            self.data['Image Controls'][key] = [imgNam,mult]
     3236
     3237    def loadControls(self,filename):
     3238        '''load controls from a .imctrl file
     3239        :param str filename: specifies a file to be read, which should end
     3240          with .imctrl
     3241        '''
     3242        File = open(filename,'r')
     3243        Slines = File.readlines()
     3244        File.close()
     3245        G2fil.LoadControls(Slines,self.data['Image Controls'])
     3246        print('file {} read into {}'.format(filename,self.name))
     3247
     3248    def saveControls(self,filename):
     3249        '''write current controls values to a .imctrl file
     3250        :param str filename: specifies a file to write, which should end
     3251          with .imctrl
     3252        '''
     3253        G2fil.WriteControls(filename,self.data['Image Controls'])
     3254        print('file {} written from {}'.format(filename,self.name))
    29433255
    29443256##########################

  • trunk/GSASIIstrIO.py

    r3813 r3814  
    1010*GSASIIstrIO: structure I/O routines*
    1111-------------------------------------
     12
     13Contains routines for reading from GPX files and printing to the .LST file.
     14Used for refinements and in G2scriptable.
     15
     16Should not contain any wxpython references as this should be able to be used
     17in non-GUI settings.
    1218
    1319'''
    … …  
    5056
    5157def GetFullGPX(GPXfile):
    52     ''' Returns complete contents of GSASII gpx file
     58    ''' Returns complete contents of GSASII gpx file.
     59    Used in :func:`GSASIIscriptable.LoadDictFromProjFile`.
    5360
    5461    :param str GPXfile: full .gpx file name
    5562    :returns: Project,nameList, where
    5663
    57       * Project (dict) is a representation of gpx file following the GSAS-II tree structure
    58         for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
    59         data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
     64      * Project (dict) is a representation of gpx file following the GSAS-II
     65        tree structure for each item: key = tree name (e.g. 'Controls',
     66        'Restraints', etc.), data is dict
    6067      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
    6168    '''

  • trunk/config_example.py

    r3759 r3814  
    184184Column_Metadata_directory = None
    185185'''When specified and when images are read, GSAS-II will read metadata from a 1-ID
    186 style .par and a .EXT_lbls (EXT = image extension) or .lbls file. See :func:`GSASIIIO.readColMetadata` for
     186style .par and a .EXT_lbls (EXT = image extension) or .lbls file. See :func:`GSASfiles.readColMetadata` for
    187187information on how this is done.
    188188'''
Note: See TracChangeset for help on using the changeset viewer.