source: trunk/GSASIIscriptable.py @ 3809

Last change on this file since 3809 was 3809, checked in by toby, 3 years ago

update scriptable to handle banks in data/inst files

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 135.9 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2019-02-03 16:35:00 +0000 (Sun, 03 Feb 2019) $
5# $Author: toby $
6# $Revision: 3809 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3809 2019-02-03 16:35:00Z toby $
9########### SVN repository information ###################
10#
11# TODO: integrate 2d data based on a model
12#
13"""
14*GSASIIscriptable: Scripting Interface*
15=======================================
16
17Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
18This file specifies several wrapper classes around GSAS-II data representations.
19They all inherit from :class:`G2ObjectWrapper`. The chief class is :class:`G2Project`,
20which represents an entire GSAS-II project and provides several methods to access
21phases, powder histograms, and execute Rietveld refinements. These routines can be
22accessed in two ways:
23the :ref:`CommandlineInterface` provides access a number of features without writing
24Python scripts of from the :ref:`API`, which allows much more versatile access from
25within Python.
26
27
28=====================
29Overview
30=====================
31The most commonly used routines are:
32
33:class:`G2Project`
34   Always needed: used to create a new project (.gpx) file or read in an existing one.
35
36:meth:`G2Project.add_powder_histogram`
37   Used to read in powder diffraction data to a project file.
38
39:meth:`G2Project.add_simulated_powder_histogram`
40   Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
41
42:meth:`G2Project.save`
43   Writes the current project to disk.
44
45:meth:`G2Project.do_refinements`
46   This is passed a list of dictionaries, where each dict defines a refinement step.
47   Passing a list with a single empty dict initiates a refinement with the current
48   parameters and flags.
49   
50:meth:`G2Project.set_refinement`
51   This is passed a single dict which is used to set parameters and flags.
52   These actions can be performed also in :meth:`G2Project.do_refinements`.
53
54Refinement dicts
55   A refinement dict sets up a single refinement step
56   (as used in :meth:`G2Project.do_refinements` and described in
57   :ref:`Project_dicts`).
58   It specifies parameter & refinement flag changes, which are usually followed by a refinement and
59   optionally by calls to locally-defined Python functions. The keys used in these dicts are
60   defined in the :ref:`Refinement_recipe` table, below.
61
62There are several ways to set parameters project using different level objects, as is
63are described in sections below, but the simplest way to access parameters and flags
64in a project is to use the above routines.
65
66=====================
67Refinement parameters
68=====================
69The most complex part of scripting GSAS-II comes in setting up the input to control refinements, which is
70described here.
71
72.. _Project_dicts:
73
74-----------------------------
75Project-level Parameter Dict
76-----------------------------
77
78As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
79which can be accessed individually by the objects that encapsulate individual phases and histograms
80but it will be usually simplest to create a composite dictionary
81that is used at the project-level. A dict is created with keys
82"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
83(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
84determine parameter values and will determine which parameters will be refined.
85
86The specific keys and subkeys that can be used are defined in tables
87:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
88
89Note that optionally a list of histograms and/or phases can be supplied in the call to
90:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
91phases and histograms.
92
93As an example:
94
95.. code-block::  python
96
97    pardict = {'set': { 'Limits': [0.8, 12.0],
98                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
99                       'Background': {'type': 'chebyschev', 'refine': True}},
100              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
101    my_project.set_refinement(pardict)
102   
103.. _Refinement_recipe:
104   
105------------------------
106Refinement recipe
107------------------------
108Building on the :ref:`Project_dicts`,
109it is possible to specify a sequence of refinement actions as a list of
110these dicts and supplying this list
111as an argument to :meth:`G2Project.do_refinements`.
112
113As an example, this code performs the same actions as in the example in the section above:
114
115.. code-block::  python
116   
117    pardict = {'set': { 'Limits': [0.8, 12.0],
118                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
119                       'Background': {'type': 'chebyschev', 'refine': True}},
120              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
121    my_project.do_refinements([pardict])
122
123However, in addition to setting a number of parameters, this example will perform a refinement as well.
124If more than one dict is specified in the list, as is done in this example,
125two refinement steps will be performed:
126
127.. code-block::  python
128
129    my_project.do_refinements([pardict,pardict1])
130
131
132The keys defined in the following table
133may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that now keys ``histograms``
134and ``phases`` are used to limit actions to specific sets of parameters within the project.
135
136========== ============================================================================
137key         explanation
138========== ============================================================================
139set                    Specifies a dict with keys and subkeys as described in the
140                       :ref:`Refinement_parameters_fmt` section. Items listed here
141                       will be set to be refined.
142clear                  Specifies a dict, as above for set, except that parameters are
143                       cleared and thus will not be refined.
144once                   Specifies a dict as above for set, except that parameters are
145                       set for the next cycle of refinement and are cleared once the
146                       refinement step is completed.
147skip                   Normally, once parameters are processed with a set/clear/once
148                       action(s), a refinement is started. If skip is defined as True
149                       (or any other value) the refinement step is not performed.
150output                 If a file name is specified for output is will be used to save
151                       the current refinement.
152histograms             Should contain a list of histogram(s) to be used for the
153                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
154                       :ref:`HAP_parameters_table`. Note that this will be
155                       ignored for :ref:`Phase_parameters_table`. Histograms may be
156                       specified as a list of strings [('PWDR ...'),...], indices
157                       [0,1,2] or as list of objects [hist1, hist2].
158phases                 Should contain a list of phase(s) to be used for the
159                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
160                       :ref:`HAP_parameters_table`. Note that this will be
161                       ignored for :ref:`Histogram_parameters_table`.
162                       Phases may be specified as a list of strings
163                       [('Phase name'),...], indices [0,1,2] or as list of objects
164                       [phase0, phase2].
165call                   Specifies a function to call after a refinement is completed.
166                       The value supplied can be the object (typically a function)
167                       that will be called or a string that will evaluate (in the
168                       namespace inside :meth:`G2Project.iter_refinements` where
169                       ``self`` references the project.)
170                       Nothing is called if this is not specified.
171callargs               Provides a list of arguments that will be passed to the function
172                       in call (if any). If call is defined and callargs is not, the
173                       current <tt>G2Project</tt> is passed as a single argument.
174========== ============================================================================
175
176An example that performs a series of refinement steps follows:
177
178.. code-block::  python
179
180    reflist = [
181            {"set": { "Limits": { "low": 0.7 },
182                      "Background": { "no. coeffs": 3,
183                                      "refine": True }}},
184            {"set": { "LeBail": True,
185                      "Cell": True }},
186            {"set": { "Sample Parameters": ["DisplaceX"]}},
187            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
188            {"set": { "Mustrain": { "type": "uniaxial",
189                                    "refine": "equatorial",
190                                    "direction": [0, 0, 1]}}},
191            {"set": { "Mustrain": { "type": "uniaxial",
192                                    "refine": "axial"}}},
193            {"clear": { "LeBail": True},
194             "set": { "Atoms": { "Mn": "X" }}},
195            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
196    my_project.do_refinements(reflist)
197   
198
199In this example, a separate refinement step will be performed for each dict in the list (since
200"skip" is not included).
201Note that in the second from last refinement step, parameters are both set and cleared.
202   
203.. _Refinement_parameters_kinds:
204
205----------------------------
206Refinement parameter types
207----------------------------
208
209Note that parameters and refinement flags used in GSAS-II fall into three classes:
210
211    * **Histogram**: There will be a set of these for each dataset loaded into a
212      project file. The parameters available depend on the type of histogram
213      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
214      include the overall scale factor, background, instrument and sample parameters;
215      see the :ref:`Histogram_parameters_table` table for a list of the histogram
216      parameters where access has been provided.
217     
218    * **Phase**: There will be a set of these for each phase loaded into a
219      project file. While some parameters are found in all types of phases,
220      others are only found in certain types (modulated, magnetic, protein...).
221      Typical phase parameters include unit cell lengths and atomic positions; see the
222      :ref:`Phase_parameters_table` table for a list of the phase     
223      parameters where access has been provided.
224     
225    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
226      that is associated with each phase, so that if there are ``N`` phases and ``M``
227      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
228      histograms are not linked to all phases.) Typical HAP parameters include the
229      phase fractions, sample microstrain and crystallite size broadening terms,
230      hydrostatic strain perturbations of the unit cell and preferred orientation
231      values.
232      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
233      been provided.
234
235.. _Refinement_parameters_fmt:
236
237=================================
238Specifying Refinement Parameters
239=================================
240
241Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
242where the details of these dicts are organized depends on the
243type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
244of keys (as described below) for each of the three types of parameters.
245
246.. _Histogram_parameters_table:
247
248--------------------
249Histogram parameters
250--------------------
251
252This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
253and :func:`G2PwdrData.clear_refinements`. As an example,
254
255.. code-block::  python
256
257   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
258                         "Sample Parameters": ["Scale"],
259                         "Limits": [10000, 40000]})
260
261With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
262``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
263key is used to define specific histograms. As an example:
264
265.. code-block::  python
266
267  gsas_proj.do_refinements([
268      {'set': {
269          'Background': {'no.coeffs': 3, 'refine': True},
270          'Sample Parameters': ['Scale'],
271          'Limits': [10000, 40000]},
272      'histograms': [1,2]}
273                            ])
274
275Note that below in the Instrument Parameters section,
276related profile parameters (such as U and V) are grouped together but
277separated by commas to save space in the table.
278
279.. tabularcolumns:: |l|l|p{3.5in}|
280
281===================== ====================  =================================================
282key                   subkey                explanation
283===================== ====================  =================================================
284Limits                                      The range of 2-theta (degrees) or TOF (in
285                                            microsec) range of values to use. Can
286                                            be either a dictionary of 'low' and/or 'high',
287                                            or a list of 2 items [low, high]
288\                     low                   Sets the low limit
289\                     high                  Sets the high limit
290
291Sample Parameters                           Should be provided as a **list** of subkeys
292                                            to set or clear, e.g. ['DisplaceX', 'Scale']
293\                     Absorption
294\                     Contrast
295\                     DisplaceX             Sample displacement along the X direction
296\                     DisplaceY             Sample displacement along the Y direction
297\                     Scale                 Histogram Scale factor
298
299Background                                  Sample background. If value is a boolean,
300                                            the background's 'refine' parameter is set
301                                            to the given boolean. Usually should be a
302                                            dictionary with any of the following keys:
303\                     type                  The background model, e.g. 'chebyschev'
304\                     refine                The value of the refine flag, boolean
305\                     no. coeffs            Number of coefficients to use, integer
306\                     coeffs                List of floats, literal values for background
307\                     FixedPoints           List of (2-theta, intensity) values for fixed points
308\                     fit fixed points      If True, triggers a fit to the fixed points to
309                                            be calculated. It is calculated when this key is
310                                            detected, regardless of calls to refine.
311
312Instrument Parameters                       As in Sample Paramters, provide as a **list** of
313                                            subkeys to
314                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
315\                     U, V, W               Gaussian peak profile terms
316\                     X, Y, Z               Lorentzian peak profile terms
317\                     alpha, beta-0,        TOF profile terms
318                      beta-1, beta-q,
319\                     sig-0, sig-1,         TOF profile terms
320                      sig-2, sig-q
321\                     difA, difB, difC      TOF Calibration constants
322\                     Zero                  Zero shift
323\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
324\                     Polariz.              Polarization parameter
325\                     Lam                   Lambda, the incident wavelength
326===================== ====================  =================================================
327
328.. _Phase_parameters_table:
329
330----------------
331Phase parameters
332----------------
333
334This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
335and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
336these parameters should be placed inside a dict with a key
337``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
338key is used to define specific phase(s).
339
340
341.. tabularcolumns:: |l|p{4.5in}|
342
343======= ==========================================================
344key                   explanation
345======= ==========================================================
346Cell                  Whether or not to refine the unit cell.
347Atoms                 Dictionary of atoms and refinement flags.
348                      Each key should be an atom label, e.g.
349                      'O3', 'Mn5', and each value should be
350                      a string defining what values to refine.
351                      Values can be any combination of 'F'
352                      for fractional occupancy, 'X' for position,
353                      and 'U' for Debye-Waller factor
354LeBail                Enables LeBail intensity extraction.
355======= ==========================================================
356
357
358.. _HAP_parameters_table:
359
360
361Histogram-and-phase parameters
362------------------------------
363
364This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
365and :func:`G2Phase.clear_HAP_refinements`. When supplied to
366:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
367``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
368unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
369
370.. tabularcolumns:: |l|l|p{3.5in}|
371
372=============  ==========  ============================================================
373key             subkey                 explanation
374=============  ==========  ============================================================
375Babinet                                Should be a **list** of the following
376                                       subkeys. If not, assumes both
377                                       BabA and BabU
378\               BabA
379\               BabU
380Extinction                             Boolean, True to refine.
381HStrain                                Boolean, True to refine all appropriate
382                                       $D_ij$ terms.
383Mustrain
384\               type                   Mustrain model. One of 'isotropic',
385                                       'uniaxial', or 'generalized'. Should always
386                                       be specified.
387\              direction               For uniaxial only. A list of three
388                                       integers,
389                                       the [hkl] direction of the axis.
390\               refine                 Usually boolean, set to True to refine.
391                                       or False to clear.
392                                       For uniaxial model, can specify a value
393                                       of 'axial' or 'equatorial' to set that flag
394                                       to True or a single
395                                       boolean sets both axial and equatorial.
396Size                                   
397\               type                   Size broadening model. One of 'isotropic',
398                                       'uniaxial', or 'ellipsoid'. Should always
399                                       be specified.
400\              direction               For uniaxial only. A list of three
401                                       integers,
402                                       the [hkl] direction of the axis.
403\               refine                 Boolean, True to refine.
404\               value                  float, size value in microns
405Pref.Ori.                              Boolean, True to refine
406Show                                   Boolean, True to refine
407Use                                    Boolean, True to refine
408Scale                                  Phase fraction; Boolean, True to refine
409=============  ==========  ============================================================
410
411------------------------
412Histogram/Phase objects
413------------------------
414Each phase and powder histogram in a :class:`G2Project` object has an associated
415object. Parameters within each individual object can be turned on and off by calling
416:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
417for histogram parameters;
418:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
419for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
420:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
421
422.. code-block::  python
423
424    params = { 'Limits': [0.8, 12.0],
425               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
426               'Background': {'type': 'chebyschev', 'refine': True}}
427    some_histogram.set_refinements(params)
428
429Likewise to turn refinement flags on, use code such as this:
430
431.. code-block::  python
432
433    params = { 'Instrument Parameters': ['U', 'V', 'W']}
434    some_histogram.set_refinements(params)
435
436and to turn these refinement flags, off use this (Note that the
437``.clear_refinements()`` methods will usually will turn off refinement even
438if a refinement parameter is set in the dict to True.):
439
440.. code-block::  python
441
442    params = { 'Instrument Parameters': ['U', 'V', 'W']}
443    some_histogram.clear_refinements(params)
444
445For phase parameters, use code such as this:
446   
447.. code-block::  python
448
449    params = { 'LeBail': True, 'Cell': True,
450               'Atoms': { 'Mn1': 'X',
451                          'O3': 'XU',
452                          'V4': 'FXU'}}
453    some_histogram.set_refinements(params)
454
455
456and here is an example for HAP parameters:
457
458.. code-block::  python
459
460    params = { 'Babinet': 'BabA',
461               'Extinction': True,
462               'Mustrain': { 'type': 'uniaxial',
463                             'direction': [0, 0, 1],
464                             'refine': True}}
465    some_phase.set_HAP_refinements(params)
466
467Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
468
469.. _CommandlineInterface:
470
471=======================================
472GSASIIscriptable Command-line Interface
473=======================================
474
475The routines described above are intended to be called from a Python script, but an
476alternate way to access some of the same functionality is to
477invoke the ``GSASIIscriptable.py`` script from
478the command line usually from within a shell script or batch file. This
479will usually be done with a command such as::
480
481       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
482
483    The following subcommands are defined:
484
485        * create, see :func:`create`
486        * add, see :func:`add`
487        * dump, see :func:`dump`
488        * refine, see :func:`refine`
489        * seqrefine, see :func:`seqrefine`
490        * export, :func:`export`
491        * browse, see :func:`IPyBrowse`
492       
493Run::
494
495   python GSASIIscriptable.py --help
496
497to show the available subcommands, and inspect each subcommand with
498`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
499
500.. _JsonFormat:
501
502-------------------------
503Parameters in JSON files
504-------------------------
505
506The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
507a JSON format file
508(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
509This dict may have two keys:
510
511refinements:
512  This defines the a set of refinement steps in a JSON representation of a
513  :ref:`Refinement_recipe` list.
514
515code:
516  This optionally defines Python code that will be executed after the project is loaded,
517  but before the refinement is started. This can be used to execute Python code to change
518  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
519  project object is accessed with variable ``proj``) or to define code that will be called
520  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
521   
522JSON website: `Introducing JSON <http://json.org/>`_.
523
524.. _API:
525
526============================================================
527GSASIIscriptable Application Layer (API)
528============================================================
529
530The large number of classes and modules in this module are described below.
531Most commonly a script will create a G2Project object using :class:`G2Project` and then
532perform actions such as
533adding a histogram (method :meth:`G2Project.add_powder_histogram`),
534adding a phase (method :meth:`G2Project.add_phase`),
535or setting parameters and performing a refinement
536(method :meth:`G2Project.do_refinements`).
537
538In some cases, it may be easier to use
539methods inside :class:`G2PwdrData` or :class:`G2Phase` or these objects
540may offer more options.
541
542---------------------------------------------------------------
543Complete Documentation: All classes and functions
544---------------------------------------------------------------
545"""
546from __future__ import division, print_function
547import argparse
548import os.path as ospath
549import datetime as dt
550import sys
551import platform
552if '2' in platform.python_version_tuple()[0]:
553    import cPickle
554    strtypes = (str,unicode)
555else:
556    import _pickle as cPickle
557    strtypes = (str,bytes)
558import imp
559import copy
560import os
561import random as ran
562
563import numpy.ma as ma
564import scipy.interpolate as si
565import numpy as np
566import scipy as sp
567
568import GSASIIpath
569GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
570import GSASIIobj as G2obj
571import GSASIIpwd as G2pwd
572import GSASIIstrMain as G2strMain
573import GSASIIspc as G2spc
574import GSASIIElem as G2elem
575
576
577# Delay imports to not slow down small scripts
578G2fil = None
579PwdrDataReaders = []
580PhaseReaders = []
581exportersByExtension = {}
582'''Specifies the list of extensions that are supported for Powder data export'''
583
584def LoadG2fil():
585    """Delay importing this module, it is slow"""
586    global G2fil
587    if G2fil is None:
588        import GSASIIfiles
589        G2fil = GSASIIfiles
590        global PwdrDataReaders
591        global PhaseReaders
592        PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
593        PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
594        AllExporters = G2fil.LoadExportRoutines(None)
595        global exportersByExtension
596        exportersByExtension = {}
597        for obj in AllExporters:
598            try:
599                obj.Writer
600            except AttributeError:
601                continue
602            for typ in obj.exporttype:
603                if typ not in exportersByExtension:
604                    exportersByExtension[typ] = {obj.extension:obj}
605                else:
606                    exportersByExtension[typ][obj.extension] = obj
607
608def LoadDictFromProjFile(ProjFile):
609    '''Read a GSAS-II project file and load items to dictionary
610   
611    :param str ProjFile: GSAS-II project (name.gpx) full file name
612    :returns: Project,nameList, where
613
614      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
615        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
616        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
617      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
618
619    Example for fap.gpx::
620
621      Project = {                 #NB:dict order is not tree order
622        u'Phases':{'data':None,'fap':{phase dict}},
623        u'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
624        u'Rigid bodies':{'data': {rigid body dict}},
625        u'Covariance':{'data':{covariance data dict}},
626        u'Controls':{'data':{controls data dict}},
627        u'Notebook':{'data':[notebook list]},
628        u'Restraints':{'data':{restraint data dict}},
629        u'Constraints':{'data':{constraint data dict}}]
630        }
631      nameList = [                #NB: reproduces tree order
632        [u'Notebook',],
633        [u'Controls',],
634        [u'Covariance',],
635        [u'Constraints',],
636        [u'Restraints',],
637        [u'Rigid bodies',],
638        [u'PWDR FAP.XRA Bank 1',
639             u'Comments',
640             u'Limits',
641             u'Background',
642             u'Instrument Parameters',
643             u'Sample Parameters',
644             u'Peak List',
645             u'Index Peak List',
646             u'Unit Cells List',
647             u'Reflection Lists'],
648        [u'Phases', u'fap']
649        ]
650    '''
651    # Let IOError be thrown if file does not exist
652    # if not ospath.exists(ProjFile):
653    #     print ('\n*** Error attempt to open project file that does not exist:\n   '+
654    #         str(ProjFile))
655    #     return
656    file = open(ProjFile,'rb')
657    # print('loading from file: {}'.format(ProjFile))
658    Project = {}
659    nameList = []
660    try:
661        while True:
662            try:
663                data = cPickle.load(file)
664            except EOFError:
665                break
666            datum = data[0]
667            Project[datum[0]] = {'data':datum[1]}
668            nameList.append([datum[0],])
669            for datus in data[1:]:
670                Project[datum[0]][datus[0]] = datus[1]
671                nameList[-1].append(datus[0])
672        file.close()
673        # print('project load successful')
674    except:
675        raise IOError("Error reading file "+str(ProjFile)+". This is not a GSAS-II .gpx file")
676    finally:
677        file.close()
678    return Project,nameList
679
680def SaveDictToProjFile(Project,nameList,ProjFile):
681    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
682
683    :param dict Project: representation of gpx file following the GSAS-II
684        tree structure as described for LoadDictFromProjFile
685    :param list nameList: names of main tree entries & subentries used to reconstruct project file
686    :param str ProjFile: full file name for output project.gpx file (including extension)
687    '''
688    file = open(ProjFile,'wb')
689    try:
690        for name in nameList:
691            data = []
692            item = Project[name[0]]
693            data.append([name[0],item['data']])
694            for item2 in name[1:]:
695                data.append([item2,item[item2]])
696            cPickle.dump(data,file,1)
697    finally:
698        file.close()
699
700# def ImportPowder(reader,filename):
701#     '''Use a reader to import a powder diffraction data file
702
703#     :param str reader: a scriptable reader
704#     :param str filename: full name of powder data file; can be "multi-Bank" data
705
706#     :returns: list rdlist: list of reader objects containing powder data, one for each
707#         "Bank" of data encountered in file. Items in reader object of interest are:
708
709#           * rd.comments: list of str: comments found on powder file
710#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
711#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
712#     '''
713#     rdfile,rdpath,descr = imp.find_module(reader)
714#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
715#     rd = rdclass.GSAS_ReaderClass()
716#     if not rd.scriptable:
717#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
718#         return None
719#     rdlist = []
720#     if rd.ContentsValidator(filename):
721#         repeat = True
722#         rdbuffer = {} # create temporary storage for file reader
723#         block = 0
724#         while repeat: # loop if the reader asks for another pass on the file
725#             block += 1
726#             repeat = False
727#             rd.objname = ospath.basename(filename)
728#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
729#             if flag:
730#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
731#                 if rd.repeat:
732#                     repeat = True
733#         return rdlist
734#     print(rd.errors)
735#     return None
736
737def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
738    '''Create an initial Histogram dictionary
739
740    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
741    '''
742    if dType in ['SXC','SNC']:
743        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
744            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
745            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
746            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
747            'Flack':[0.0,False]}
748    elif dType == 'SNT':
749        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
750            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
751            'Extinction':['Lorentzian','None', {
752            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
753    elif 'P' in dType:
754        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
755            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
756            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
757                [1.,1.,1.,0.,0.,0.],6*[False,]],
758            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
759                NShkl*[0.01,],NShkl*[False,]],
760            'HStrain':[NDij*[0.0,],NDij*[False,]],
761            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
762
763
764def PreSetup(data):
765    '''Create part of an initial (empty) phase dictionary
766
767    from GSASIIphsGUI.py, near end of UpdatePhaseData
768
769    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
770    '''
771    if 'RBModels' not in data:
772        data['RBModels'] = {}
773    if 'MCSA' not in data:
774        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
775    if 'dict' in str(type(data['MCSA']['Results'])):
776        data['MCSA']['Results'] = []
777    if 'Modulated' not in data['General']:
778        data['General']['Modulated'] = False
779#    if 'modulated' in data['General']['Type']:
780#        data['General']['Modulated'] = True
781#        data['General']['Type'] = 'nuclear'
782
783
784def SetupGeneral(data, dirname):
785    """Helps initialize phase data.
786
787    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
788
789    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
790    """
791    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
792                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
793    generalData = data['General']
794    atomData = data['Atoms']
795    generalData['AtomTypes'] = []
796    generalData['Isotopes'] = {}
797
798    if 'Isotope' not in generalData:
799        generalData['Isotope'] = {}
800    if 'Data plot type' not in generalData:
801        generalData['Data plot type'] = 'Mustrain'
802    if 'POhkl' not in generalData:
803        generalData['POhkl'] = [0,0,1]
804    if 'Map' not in generalData:
805        generalData['Map'] = mapDefault.copy()
806    if 'Flip' not in generalData:
807        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
808            'k-factor':0.1,'k-Max':20.,}
809    if 'testHKL' not in generalData['Flip']:
810        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
811    if 'doPawley' not in generalData:
812        generalData['doPawley'] = False     #ToDo: change to ''
813    if 'Pawley dmin' not in generalData:
814        generalData['Pawley dmin'] = 1.0
815    if 'Pawley neg wt' not in generalData:
816        generalData['Pawley neg wt'] = 0.0
817    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
818        'MCSA controls' not in generalData:
819        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
820        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
821        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
822    if 'AtomPtrs' not in generalData:
823        generalData['AtomPtrs'] = [3,1,7,9]
824        if generalData['Type'] == 'macromolecular':
825            generalData['AtomPtrs'] = [6,4,10,12]
826        elif generalData['Type'] == 'magnetic':
827            generalData['AtomPtrs'] = [3,1,10,12]
828    if generalData['Modulated']:
829        generalData['Type'] = 'nuclear'
830        if 'Super' not in generalData:
831            generalData['Super'] = 1
832            generalData['SuperVec'] = [[0,0,.1],False,4]
833            generalData['SSGData'] = {}
834        if '4DmapData' not in generalData:
835            generalData['4DmapData'] = mapDefault.copy()
836            generalData['4DmapData'].update({'MapType':'Fobs'})
837    if 'Modulated' not in generalData:
838        generalData['Modulated'] = False
839    if 'HydIds' not in generalData:
840        generalData['HydIds'] = {}
841    cx,ct,cs,cia = generalData['AtomPtrs']
842    generalData['NoAtoms'] = {}
843    generalData['BondRadii'] = []
844    generalData['AngleRadii'] = []
845    generalData['vdWRadii'] = []
846    generalData['AtomMass'] = []
847    generalData['Color'] = []
848    if generalData['Type'] == 'magnetic':
849        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
850        landeg = generalData.get('Lande g',[])
851    generalData['Mydir'] = dirname
852    badList = {}
853    for iat,atom in enumerate(atomData):
854        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
855        if generalData['AtomTypes'].count(atom[ct]):
856            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
857        elif atom[ct] != 'UNK':
858            Info = G2elem.GetAtomInfo(atom[ct])
859            if not Info:
860                if atom[ct] not in badList:
861                    badList[atom[ct]] = 0
862                badList[atom[ct]] += 1
863                atom[ct] = 'UNK'
864                continue
865            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
866            generalData['AtomTypes'].append(atom[ct])
867            generalData['Z'] = Info['Z']
868            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
869            generalData['BondRadii'].append(Info['Drad'])
870            generalData['AngleRadii'].append(Info['Arad'])
871            generalData['vdWRadii'].append(Info['Vdrad'])
872            if atom[ct] in generalData['Isotope']:
873                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
874                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
875                    generalData['Isotope'][atom[ct]] = isotope
876                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
877            else:
878                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
879                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
880                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
881                    generalData['Isotope'][atom[ct]] = isotope
882                generalData['AtomMass'].append(Info['Mass'])
883            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
884            generalData['Color'].append(Info['Color'])
885            if generalData['Type'] == 'magnetic':
886                if len(landeg) < len(generalData['AtomTypes']):
887                    landeg.append(2.0)
888    if generalData['Type'] == 'magnetic':
889        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
890
891    if badList:
892        msg = 'Warning: element symbol(s) not found:'
893        for key in badList:
894            msg += '\n\t' + key
895            if badList[key] > 1:
896                msg += ' (' + str(badList[key]) + ' times)'
897        raise G2ScriptException("Phase error:\n" + msg)
898        # wx.MessageBox(msg,caption='Element symbol error')
899    F000X = 0.
900    F000N = 0.
901    for i,elem in enumerate(generalData['AtomTypes']):
902        F000X += generalData['NoAtoms'][elem]*generalData['Z']
903        isotope = generalData['Isotope'][elem]
904        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
905    generalData['F000X'] = F000X
906    generalData['F000N'] = F000N
907    import GSASIImath as G2mth
908    generalData['Mass'] = G2mth.getMass(generalData)
909
910
911def make_empty_project(author=None, filename=None):
912    """Creates an dictionary in the style of GSASIIscriptable, for an empty
913    project.
914
915    If no author name or filename is supplied, 'no name' and
916    <current dir>/test_output.gpx are used , respectively.
917
918    Returns: project dictionary, name list
919
920    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
921    """
922    if not filename:
923        filename = 'test_output.gpx'
924    filename = os.path.abspath(filename)
925    gsasii_version = str(GSASIIpath.GetVersionNumber())
926    LoadG2fil()
927    import matplotlib as mpl
928    python_library_versions = G2fil.get_python_versions([mpl, np, sp])
929
930    controls_data = dict(G2obj.DefaultControls)
931    controls_data['LastSavedAs'] = filename
932    controls_data['LastSavedUsing'] = gsasii_version
933    controls_data['PythonVersions'] = python_library_versions
934    if author:
935        controls_data['Author'] = author
936
937    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
938                                       'Global': []}},
939              'Controls': {'data': controls_data},
940              u'Covariance': {'data': {}},
941              u'Notebook': {'data': ['']},
942              u'Restraints': {'data': {}},
943              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
944                                'Residue': {'AtInfo': {}},
945                                'Vector':  {'AtInfo': {}}}}}
946
947    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
948             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
949
950    return output, names
951
952
953class G2ImportException(Exception):
954    pass
955
956class G2ScriptException(Exception):
957    pass
958
959def import_generic(filename, readerlist, fmthint=None, bank=None):
960    """Attempt to import a filename, using a list of reader objects.
961
962    Returns the first reader object which worked."""
963    # Translated from OnImportGeneric method in GSASII.py
964    primaryReaders, secondaryReaders = [], []
965    for reader in readerlist:
966        if fmthint is not None and fmthint not in reader.formatName: continue
967        flag = reader.ExtensionValidator(filename)
968        if flag is None:
969            secondaryReaders.append(reader)
970        elif flag:
971            primaryReaders.append(reader)
972    if not secondaryReaders and not primaryReaders:
973        raise G2ImportException("Could not read file: ", filename)
974
975    with open(filename, 'Ur') as fp:
976        rd_list = []
977
978        for rd in primaryReaders + secondaryReaders:
979            # Initialize reader
980            rd.selections = []
981            if bank is None:
982                rd.selections = []
983            else:
984                rd.selections = [bank-1]
985            rd.dnames = []
986            rd.ReInitialize()
987            # Rewind file
988            rd.errors = ""
989            if not rd.ContentsValidator(filename):
990                # Report error
991                print("Warning: File {} has a validation error, continuing".format(filename))
992            if len(rd.selections) > 1:
993                raise G2ImportException("File {} has {} banks. Specify which bank to read with databank param."
994                                .format(filename,len(rd.selections)))
995
996            block = 0
997            rdbuffer = {}
998            repeat = True
999            while repeat:
1000                repeat = False
1001                block += 1
1002                rd.objname = os.path.basename(filename)
1003                try:
1004                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
1005                except:
1006                    flag = False
1007                if flag:
1008                    # Omitting image loading special cases
1009                    rd.readfilename = filename
1010                    rd_list.append(copy.deepcopy(rd))
1011                    repeat = rd.repeat
1012                else:
1013                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
1014            if rd_list:
1015                if rd.warnings:
1016                    print("Read warning by", rd.formatName, "reader:",
1017                          rd.warnings, file=sys.stderr)
1018                elif bank is None:
1019                    print("{} read by Reader {}"
1020                              .format(filename,rd.formatName))
1021                else:
1022                    print("{} block # {} read by Reader {}"
1023                              .format(filename,bank,rd.formatName))
1024                return rd_list
1025    raise G2ImportException("No reader could read file: " + filename)
1026
1027
1028def load_iprms(instfile, reader, bank=None):
1029    """Loads instrument parameters from a file, and edits the
1030    given reader.
1031
1032    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1033    """
1034    LoadG2fil()
1035    ext = os.path.splitext(instfile)[1]
1036
1037    if ext.lower() == '.instprm':
1038        # New GSAS File, load appropriately
1039        # this likely needs some work as there can now be more than one bank
1040        with open(instfile) as f:
1041            lines = f.readlines()
1042        bank = reader.powderentry[2] # this may not be the right entry to use
1043        numbanks = reader.numbanks
1044        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1045
1046        reader.instfile = instfile
1047        reader.instmsg = 'GSAS-II file' + instfile
1048        return iparms
1049    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1050        raise ValueError('Expected .prm file, found: ', instfile)
1051
1052    # It's an old GSAS file, load appropriately
1053    Iparm = {}
1054    with open(instfile, 'Ur') as fp:
1055        for line in fp:
1056            if '#' in line:
1057                continue
1058            Iparm[line[:12]] = line[12:-1]
1059    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1060    if bank is not None:
1061        # pull out requested bank # bank from the data, and change the bank to 1
1062        Iparm,IparmC = {},Iparm
1063        for key in IparmC:
1064            if 'INS' not in key[:3]: continue   #skip around rubbish lines in some old iparm
1065            if key[4:6] == "  ":
1066                Iparm[key] = IparmC[key]
1067            elif int(key[4:6].strip()) == bank:
1068                Iparm[key[:4]+' 1'+key[6:]] = IparmC[key]           
1069        reader.instbank = bank
1070    elif ibanks == 1:
1071        reader.instbank = 1
1072    else: 
1073        raise G2ImportException("Instrument parameter file has {} banks, select one with instbank param."
1074                                    .format(ibanks))
1075    reader.powderentry[2] = 1
1076    reader.instfile = instfile
1077    reader.instmsg = instfile + ' bank ' + str(reader.instbank)
1078    return G2fil.SetPowderInstParms(Iparm, reader)
1079
1080def load_pwd_from_reader(reader, instprm, existingnames=[],bank=None):
1081    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1082
1083    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1084
1085    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1086    """
1087    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1088    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1089
1090    try:
1091        Iparm1, Iparm2 = instprm
1092    except ValueError:
1093        Iparm1, Iparm2 = load_iprms(instprm, reader, bank=bank)
1094        print('Instrument parameters read:',reader.instmsg)
1095    Ymin = np.min(reader.powderdata[1])
1096    Ymax = np.max(reader.powderdata[1])
1097    valuesdict = {'wtFactor': 1.0,
1098                  'Dummy': False,
1099                  'ranId': ran.randint(0, sys.maxsize),
1100                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1101                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1102                  'Yminmax': [Ymin, Ymax]}
1103    reader.Sample['ranId'] = valuesdict['ranId']
1104
1105    # Ending keys:
1106    # [u'Reflection Lists',
1107    #  u'Limits',
1108    #  'data',
1109    #  u'Index Peak List',
1110    #  u'Comments',
1111    #  u'Unit Cells List',
1112    #  u'Sample Parameters',
1113    #  u'Peak List',
1114    #  u'Background',
1115    #  u'Instrument Parameters']
1116    Tmin = np.min(reader.powderdata[0])
1117    Tmax = np.max(reader.powderdata[0])
1118
1119    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1120                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1121
1122    output_dict = {u'Reflection Lists': {},
1123                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1124                   u'data': [valuesdict, reader.powderdata, HistName],
1125                   u'Index Peak List': [[], []],
1126                   u'Comments': reader.comments,
1127                   u'Unit Cells List': [],
1128                   u'Sample Parameters': reader.Sample,
1129                   u'Peak List': {'peaks': [], 'sigDict': {}},
1130                   u'Background': reader.pwdparms.get('Background', default_background),
1131                   u'Instrument Parameters': [Iparm1, Iparm2],
1132                   }
1133
1134    names = [u'Comments',
1135             u'Limits',
1136             u'Background',
1137             u'Instrument Parameters',
1138             u'Sample Parameters',
1139             u'Peak List',
1140             u'Index Peak List',
1141             u'Unit Cells List',
1142             u'Reflection Lists']
1143
1144    # TODO controls?? GSASII.py:1664-7
1145
1146    return HistName, [HistName] + names, output_dict
1147
1148
1149def _deep_copy_into(from_, into):
1150    """Helper function for reloading .gpx file. See G2Project.reload()
1151
1152    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1153    """
1154    if isinstance(from_, dict) and isinstance(into, dict):
1155        combined_keys = set(from_.keys()).union(into.keys())
1156        for key in combined_keys:
1157            if key in from_ and key in into:
1158                both_dicts = (isinstance(from_[key], dict)
1159                              and isinstance(into[key], dict))
1160                both_lists = (isinstance(from_[key], list)
1161                              and isinstance(into[key], list))
1162                if both_dicts or both_lists:
1163                    _deep_copy_into(from_[key], into[key])
1164                else:
1165                    into[key] = from_[key]
1166            elif key in from_:
1167                into[key] = from_[key]
1168            else:  # key in into
1169                del into[key]
1170    elif isinstance(from_, list) and isinstance(into, list):
1171        if len(from_) == len(into):
1172            for i in range(len(from_)):
1173                both_dicts = (isinstance(from_[i], dict)
1174                              and isinstance(into[i], dict))
1175                both_lists = (isinstance(from_[i], list)
1176                              and isinstance(into[i], list))
1177                if both_dicts or both_lists:
1178                    _deep_copy_into(from_[i], into[i])
1179                else:
1180                    into[i] = from_[i]
1181        else:
1182            into[:] = from_
1183
1184
1185class G2ObjectWrapper(object):
1186    """Base class for all GSAS-II object wrappers.
1187
1188    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1189    of overrides are implemented so that the wrapper behaves like a dictionary.
1190
1191    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1192    """
1193    def __init__(self, datadict):
1194        self.data = datadict
1195
1196    def __getitem__(self, key):
1197        return self.data[key]
1198
1199    def __setitem__(self, key, value):
1200        self.data[key] = value
1201
1202    def __contains__(self, key):
1203        return key in self.data
1204
1205    def get(self, k, d=None):
1206        return self.data.get(k, d)
1207
1208    def keys(self):
1209        return self.data.keys()
1210
1211    def values(self):
1212        return self.data.values()
1213
1214    def items(self):
1215        return self.data.items()
1216
1217
1218class G2Project(G2ObjectWrapper):
1219    """
1220    Represents an entire GSAS-II project.
1221
1222    :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1223            creates an empty project.
1224    :param str author: Author's name (not yet implemented)
1225    :param str newgpx: The filename the project should be saved to in
1226            the future. If both newgpx and gpxfile are present, the project is
1227            loaded from the gpxfile, then when saved will be written to newgpx.
1228    :param str filename: Name to be used to save the project. Has same function as
1229            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1230            is preferred over filename.
1231
1232    There are two ways to initialize this object:
1233
1234    >>> # Load an existing project file
1235    >>> proj = G2Project('filename.gpx')
1236   
1237    >>> # Create a new project
1238    >>> proj = G2Project(newgpx='new_file.gpx')
1239   
1240    Histograms can be accessed easily.
1241
1242    >>> # By name
1243    >>> hist = proj.histogram('PWDR my-histogram-name')
1244   
1245    >>> # Or by index
1246    >>> hist = proj.histogram(0)
1247    >>> assert hist.id == 0
1248   
1249    >>> # Or by random id
1250    >>> assert hist == proj.histogram(hist.ranId)
1251
1252    Phases can be accessed the same way.
1253
1254    >>> phase = proj.phase('name of phase')
1255
1256    New data can also be loaded via :meth:`~G2Project.add_phase` and
1257    :meth:`~G2Project.add_powder_histogram`.
1258
1259    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1260                                         'instrument_parameters.prm')
1261    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1262
1263    Parameters for Rietveld refinement can be turned on and off as well.
1264    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1265    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1266    """
1267    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1268        if filename is not None and newgpx is not None:
1269            raise G2ScriptException('Do not use filename and newgpx together')
1270        elif newgpx is not None:
1271            filename = newgpx
1272        if gpxfile is None:
1273            filename = os.path.abspath(os.path.expanduser(filename))
1274            self.filename = filename
1275            self.data, self.names = make_empty_project(author=author, filename=filename)
1276        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1277                                       # filename is valid, etc.
1278            if isinstance(filename, str): 
1279                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1280            else: 
1281                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1282            # TODO set author
1283            self.data, self.names = LoadDictFromProjFile(gpxfile)
1284            self.update_ids()
1285        else:
1286            raise ValueError("Not sure what to do with gpxfile")
1287
1288    @classmethod
1289    def from_dict_and_names(cls, gpxdict, names, filename=None):
1290        """Creates a :class:`G2Project` directly from
1291        a dictionary and a list of names. If in doubt, do not use this.
1292
1293        :returns: a :class:`G2Project`
1294        """
1295        out = cls()
1296        if filename:
1297            filename = os.path.abspath(os.path.expanduser(filename))
1298            out.filename = filename
1299            gpxdict['Controls']['data']['LastSavedAs'] = filename
1300        else:
1301            try:
1302                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1303            except KeyError:
1304                out.filename = None
1305        out.data = gpxdict
1306        out.names = names
1307
1308    def save(self, filename=None):
1309        """Saves the project, either to the current filename, or to a new file.
1310
1311        Updates self.filename if a new filename provided"""
1312        # TODO update LastSavedUsing ?
1313        if filename:
1314            filename = os.path.abspath(os.path.expanduser(filename))
1315            self.data['Controls']['data']['LastSavedAs'] = filename
1316            self.filename = filename
1317        elif not self.filename:
1318            raise AttributeError("No file name to save to")
1319        SaveDictToProjFile(self.data, self.names, self.filename)
1320
1321    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None,
1322                                 databank=None, instbank=None):
1323        """Loads a powder data histogram into the project.
1324
1325        Automatically checks for an instrument parameter file, or one can be
1326        provided. Note that in unix fashion, "~" can be used to indicate the
1327        home directory (e.g. ~/G2data/data.fxye).
1328
1329        :param str datafile: The powder data file to read, a filename.
1330        :param str iparams: The instrument parameters file, a filename.
1331        :param list phases: Phases to link to the new histogram
1332        :param str fmthint: If specified, only importers where the format name
1333          (reader.formatName, as shown in Import menu) contains the
1334          supplied string will be tried as importers. If not specified, all
1335          importers consistent with the file extension will be tried
1336          (equivalent to "guess format" in menu).
1337        :param int databank: Specifies a dataset number to read, if file contains
1338          more than set of data. This should be 1 to read the first bank in
1339          the file (etc.) regardless of the number on the Bank line, etc.
1340          Default is None which means there should only be one dataset in the
1341          file.
1342        :param int instbank: Specifies an instrument parameter set to read, if
1343          the instrument parameter file contains more than set of parameters.
1344          This will match the INS # in an GSAS type file so it will typically
1345          be 1 to read the first parameter set in the file (etc.)
1346          Default is None which means there should only be one parameter set
1347          in the file.
1348
1349        :returns: A :class:`G2PwdrData` object representing
1350            the histogram
1351        """
1352        LoadG2fil()
1353        datafile = os.path.abspath(os.path.expanduser(datafile))
1354        iparams = os.path.abspath(os.path.expanduser(iparams))
1355        pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint,bank=databank)
1356        histname, new_names, pwdrdata = load_pwd_from_reader(
1357                                          pwdrreaders[0], iparams,
1358                                          [h.name for h in self.histograms()],bank=instbank)
1359        if histname in self.data:
1360            print("Warning - redefining histogram", histname)
1361        elif self.names[-1][0] == 'Phases':
1362            self.names.insert(-1, new_names)
1363        else:
1364            self.names.append(new_names)
1365        self.data[histname] = pwdrdata
1366        self.update_ids()
1367
1368        for phase in phases:
1369            phase = self.phase(phase)
1370            self.link_histogram_phase(histname, phase)
1371
1372        return self.histogram(histname)
1373
1374    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1375                                       wavelength=None, scale=None, phases=[]):
1376        """Loads a powder data histogram into the project.
1377
1378        Requires an instrument parameter file.
1379        Note that in unix fashion, "~" can be used to indicate the
1380        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1381        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1382        as the instrument type.
1383
1384        :param str histname: A name for the histogram to be created.
1385        :param str iparams: The instrument parameters file, a filename.
1386        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1387        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1388        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1389        :param float wavelength: Wavelength for CW instruments, overriding the value
1390           in the instrument parameters file if specified.
1391        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1392           simulated noise is added to the pattern, so that if the maximum intensity is
1393           small, the noise will mask the computed pattern. The scale
1394           needs to be a large number for CW neutrons.
1395           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1396        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1397           all defined phases.
1398
1399        :returns: A :class:`G2PwdrData` object representing the histogram
1400        """
1401        LoadG2fil()
1402        iparams = os.path.abspath(os.path.expanduser(iparams))
1403        if not os.path.exists(iparams):
1404            raise G2ScriptException("File does not exist:"+iparams)
1405        rd = G2obj.ImportPowderData( # Initialize a base class reader
1406            extensionlist=tuple(),
1407            strictExtension=False,
1408            formatName = 'Simulate dataset',
1409            longFormatName = 'Compute a simulated pattern')
1410        rd.powderentry[0] = '' # no filename
1411        rd.powderentry[2] = 1 # only one bank
1412        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1413        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1414        if Tmax < Tmin:
1415            Tmin,Tmax = Tmax,Tmin
1416        Tstep = abs(Tstep)
1417        if 'TOF' in rd.idstring:
1418                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1419                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1420                N = len(x)
1421        else:           
1422                N = int((Tmax-Tmin)/Tstep)+1
1423                x = np.linspace(Tmin,Tmax,N,True)
1424                N = len(x)
1425        if N < 3:
1426            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1427        rd.powderdata = [
1428            np.array(x), # x-axis values
1429            np.zeros_like(x), # powder pattern intensities
1430            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1431            np.zeros_like(x), # calc. intensities (zero)
1432            np.zeros_like(x), # calc. background (zero)
1433            np.zeros_like(x), # obs-calc profiles
1434            ]
1435        Tmin = rd.powderdata[0][0]
1436        Tmax = rd.powderdata[0][-1]
1437        rd.idstring = histname
1438        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1439                                                            [h.name for h in self.histograms()])
1440        if histname in self.data:
1441            print("Warning - redefining histogram", histname)
1442        elif self.names[-1][0] == 'Phases':
1443            self.names.insert(-1, new_names)
1444        else:
1445            self.names.append(new_names)
1446        if scale is not None:
1447            pwdrdata['Sample Parameters']['Scale'][0] = scale
1448        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1449            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1450        self.data[histname] = pwdrdata
1451        self.update_ids()
1452
1453        for phase in phases:
1454            phase = self.phase(phase)
1455            self.link_histogram_phase(histname, phase)
1456
1457        return self.histogram(histname)
1458   
1459    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1460        """Loads a phase into the project from a .cif file
1461
1462        :param str phasefile: The CIF file from which to import the phase.
1463        :param str phasename: The name of the new phase, or None for the default
1464        :param list histograms: The names of the histograms to associate with
1465            this phase. Use proj.Histograms() to add to all histograms.
1466        :param str fmthint: If specified, only importers where the format name
1467          (reader.formatName, as shown in Import menu) contains the
1468          supplied string will be tried as importers. If not specified, all
1469          importers consistent with the file extension will be tried
1470          (equivalent to "guess format" in menu).
1471
1472        :returns: A :class:`G2Phase` object representing the
1473            new phase.
1474        """
1475        LoadG2fil()
1476        histograms = [self.histogram(h).name for h in histograms]
1477        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1478
1479        # TODO handle multiple phases in a file
1480        phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
1481        phasereader = phasereaders[0]
1482       
1483        phasename = phasename or phasereader.Phase['General']['Name']
1484        phaseNameList = [p.name for p in self.phases()]
1485        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1486        phasereader.Phase['General']['Name'] = phasename
1487
1488        if 'Phases' not in self.data:
1489            self.data[u'Phases'] = { 'data': None }
1490        assert phasename not in self.data['Phases'], "phase names should be unique"
1491        self.data['Phases'][phasename] = phasereader.Phase
1492
1493        if phasereader.Constraints:
1494            Constraints = self.data['Constraints']
1495            for i in phasereader.Constraints:
1496                if isinstance(i, dict):
1497                    if '_Explain' not in Constraints:
1498                        Constraints['_Explain'] = {}
1499                    Constraints['_Explain'].update(i)
1500                else:
1501                    Constraints['Phase'].append(i)
1502
1503        data = self.data['Phases'][phasename]
1504        generalData = data['General']
1505        SGData = generalData['SGData']
1506        NShkl = len(G2spc.MustrainNames(SGData))
1507        NDij = len(G2spc.HStrainNames(SGData))
1508        Super = generalData.get('Super', 0)
1509        if Super:
1510            SuperVec = np.array(generalData['SuperVec'][0])
1511        else:
1512            SuperVec = []
1513        UseList = data['Histograms']
1514
1515        for hist in histograms:
1516            self.link_histogram_phase(hist, phasename)
1517
1518        for obj in self.names:
1519            if obj[0] == 'Phases':
1520                phasenames = obj
1521                break
1522        else:
1523            phasenames = [u'Phases']
1524            self.names.append(phasenames)
1525        phasenames.append(phasename)
1526
1527        # TODO should it be self.filename, not phasefile?
1528        SetupGeneral(data, os.path.dirname(phasefile))
1529        self.index_ids()
1530
1531        self.update_ids()
1532        return self.phase(phasename)
1533
1534    def link_histogram_phase(self, histogram, phase):
1535        """Associates a given histogram and phase.
1536
1537        .. seealso::
1538
1539            :meth:`G2Project.histogram`
1540            :meth:`G2Project.phase`"""
1541        hist = self.histogram(histogram)
1542        phase = self.phase(phase)
1543
1544        generalData = phase['General']
1545
1546        if hist.name.startswith('HKLF '):
1547            raise NotImplementedError("HKLF not yet supported")
1548        elif hist.name.startswith('PWDR '):
1549            hist['Reflection Lists'][generalData['Name']] = {}
1550            UseList = phase['Histograms']
1551            SGData = generalData['SGData']
1552            NShkl = len(G2spc.MustrainNames(SGData))
1553            NDij = len(G2spc.HStrainNames(SGData))
1554            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1555            UseList[hist.name]['hId'] = hist.id
1556            for key, val in [('Use', True), ('LeBail', False),
1557                             ('newLeBail', True),
1558                             ('Babinet', {'BabA': [0.0, False],
1559                                          'BabU': [0.0, False]})]:
1560                if key not in UseList[hist.name]:
1561                    UseList[hist.name][key] = val
1562        else:
1563            raise RuntimeError("Unexpected histogram" + hist.name)
1564
1565
1566    def reload(self):
1567        """Reload self from self.filename"""
1568        data, names = LoadDictFromProjFile(self.filename)
1569        self.names = names
1570        # Need to deep copy the new data file data into the current tree,
1571        # so that any existing G2Phase, or G2PwdrData objects will still be
1572        # valid
1573        _deep_copy_into(from_=data, into=self.data)
1574
1575    def refine(self, newfile=None, printFile=None, makeBack=False):
1576        # TODO migrate to RefineCore
1577        # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1578        #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1579        # index_ids will automatically save the project
1580        self.index_ids()
1581        # TODO G2strMain does not properly use printFile
1582        G2strMain.Refine(self.filename, makeBack=makeBack)
1583        # Reload yourself
1584        self.reload()
1585
1586    def histogram(self, histname):
1587        """Returns the histogram named histname, or None if it does not exist.
1588
1589        :param histname: The name of the histogram (str), or ranId or index.
1590        :returns: A :class:`G2PwdrData` object, or None if
1591            the histogram does not exist
1592
1593        .. seealso::
1594            :meth:`G2Project.histograms`
1595            :meth:`G2Project.phase`
1596            :meth:`G2Project.phases`
1597            """
1598        if isinstance(histname, G2PwdrData):
1599            if histname.proj == self:
1600                return histname
1601        if histname in self.data:
1602            return G2PwdrData(self.data[histname], self)
1603        try:
1604            # see if histname is an id or ranId
1605            histname = int(histname)
1606        except ValueError:
1607            return
1608
1609        for histogram in self.histograms():
1610            if histogram.id == histname or histogram.ranId == histname:
1611                return histogram
1612
1613    def histograms(self):
1614        """Return a list of all histograms, as
1615        :class:`G2PwdrData` objects
1616
1617        .. seealso::
1618            :meth:`G2Project.histograms`
1619            :meth:`G2Project.phase`
1620            :meth:`G2Project.phases`
1621            """
1622        output = []
1623        # loop through each tree entry. If it is more than one level (more than one item in the
1624        # list of names) then it must be a histogram, unless labeled Phases or Restraints
1625        for obj in self.names:
1626            if len(obj) > 1 and obj[0] != u'Phases' and obj[0] != u'Restraints':
1627                output.append(self.histogram(obj[0]))
1628        return output
1629
1630    def phase(self, phasename):
1631        """
1632        Gives an object representing the specified phase in this project.
1633
1634        :param str phasename: A reference to the desired phase. Either the phase
1635            name (str), the phase's ranId, the phase's index (both int) or
1636            a phase object (:class:`G2Phase`)
1637        :returns: A :class:`G2Phase` object
1638        :raises: KeyError
1639
1640        .. seealso::
1641            :meth:`G2Project.histograms`
1642            :meth:`G2Project.phase`
1643            :meth:`G2Project.phases`
1644            """
1645        if isinstance(phasename, G2Phase):
1646            if phasename.proj == self:
1647                return phasename
1648        phases = self.data['Phases']
1649        if phasename in phases:
1650            return G2Phase(phases[phasename], phasename, self)
1651
1652        try:
1653            # phasename should be phase index or ranId
1654            phasename = int(phasename)
1655        except ValueError:
1656            return
1657
1658        for phase in self.phases():
1659            if phase.id == phasename or phase.ranId == phasename:
1660                return phase
1661
1662    def phases(self):
1663        """
1664        Returns a list of all the phases in the project.
1665
1666        :returns: A list of :class:`G2Phase` objects
1667
1668        .. seealso::
1669            :meth:`G2Project.histogram`
1670            :meth:`G2Project.histograms`
1671            :meth:`G2Project.phase`
1672            """
1673        for obj in self.names:
1674            if obj[0] == 'Phases':
1675                return [self.phase(p) for p in obj[1:]]
1676        return []
1677
1678    def update_ids(self):
1679        """Makes sure all phases and histograms have proper hId and pId"""
1680        # Translated from GetUsedHistogramsAndPhasesfromTree,
1681        #   GSASIIdataGUI.py:4107
1682        for i, h in enumerate(self.histograms()):
1683            h.id = i
1684        for i, p in enumerate(self.phases()):
1685            p.id = i
1686
1687    def do_refinements(self, refinements, histogram='all', phase='all',
1688                       outputnames=None, makeBack=False):
1689        """Conducts one or a series of refinements according to the
1690           input provided in parameter refinements. This is a wrapper
1691           around :meth:`iter_refinements`
1692
1693        :param list refinements: A list of dictionaries specifiying changes to be made to
1694            parameters before refinements are conducted.
1695            See the :ref:`Refinement_recipe` section for how this is defined.
1696        :param str histogram: Name of histogram for refinements to be applied
1697            to, or 'all'; note that this can be overridden for each refinement
1698            step via a "histograms" entry in the dict.
1699        :param str phase: Name of phase for refinements to be applied to, or
1700            'all'; note that this can be overridden for each refinement
1701            step via a "phases" entry in the dict.
1702        :param list outputnames: Provides a list of project (.gpx) file names
1703            to use for each refinement step (specifying None skips the save step).
1704            See :meth:`save`.
1705            Note that this can be overridden using an "output" entry in the dict.
1706        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
1707            before a refinement is performed. The default is False.
1708           
1709        To perform a single refinement without changing any parameters, use this
1710        call:
1711
1712        .. code-block::  python
1713       
1714            my_project.do_refinements([])
1715        """
1716       
1717        for proj in self.iter_refinements(refinements, histogram, phase,
1718                                          outputnames, makeBack):
1719            pass
1720        return self
1721
1722    def iter_refinements(self, refinements, histogram='all', phase='all',
1723                         outputnames=None, makeBack=False):
1724        """Conducts a series of refinements, iteratively. Stops after every
1725        refinement and yields this project, to allow error checking or
1726        logging of intermediate results. Parameter use is the same as for
1727        :meth:`do_refinements` (which calls this method).
1728
1729        >>> def checked_refinements(proj):
1730        ...     for p in proj.iter_refinements(refs):
1731        ...         # Track intermediate results
1732        ...         log(p.histogram('0').residuals)
1733        ...         log(p.phase('0').get_cell())
1734        ...         # Check if parameter diverged, nonsense answer, or whatever
1735        ...         if is_something_wrong(p):
1736        ...             raise Exception("I need a human!")
1737
1738           
1739        """
1740        if outputnames:
1741            if len(refinements) != len(outputnames):
1742                raise ValueError("Should have same number of outputs to"
1743                                 "refinements")
1744        else:
1745            outputnames = [None for r in refinements]
1746
1747        for output, refinedict in zip(outputnames, refinements):
1748            if 'histograms' in refinedict:
1749                hist = refinedict['histograms']
1750            else:
1751                hist = histogram
1752            if 'phases' in refinedict:
1753                ph = refinedict['phases']
1754            else:
1755                ph = phase
1756            if 'output' in refinedict:
1757                output = refinedict['output']
1758            self.set_refinement(refinedict, hist, ph)
1759            # Handle 'once' args - refinements that are disabled after this
1760            # refinement
1761            if 'once' in refinedict:
1762                temp = {'set': refinedict['once']}
1763                self.set_refinement(temp, hist, ph)
1764
1765            if output:
1766                self.save(output)
1767
1768            if 'skip' not in refinedict:
1769                self.refine(makeBack=makeBack)
1770            yield self
1771
1772            # Handle 'once' args - refinements that are disabled after this
1773            # refinement
1774            if 'once' in refinedict:
1775                temp = {'clear': refinedict['once']}
1776                self.set_refinement(temp, hist, ph)
1777            if 'call' in refinedict:
1778                fxn = refinedict['call']
1779                if callable(fxn):
1780                    fxn(*refinedict.get('callargs',[self]))
1781                elif callable(eval(fxn)):
1782                    eval(fxn)(*refinedict.get('callargs',[self]))
1783                else:
1784                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
1785
1786    def set_refinement(self, refinement, histogram='all', phase='all'):
1787        """Apply specified refinements to a given histogram(s) or phase(s).
1788
1789        :param dict refinement: The refinements to be conducted
1790        :param histogram: Specifies either 'all' (default), a single histogram or
1791          a list of histograms. Histograms may be specified as histogram objects
1792          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
1793          of the histogram in the project, numbered starting from 0.
1794          Omitting the parameter or the string 'all' indicates that parameters in
1795          all histograms should be set.
1796        :param phase: Specifies either 'all' (default), a single phase or
1797          a list of phases. Phases may be specified as phase objects
1798          (see :class:`G2Phase`), the phase name (str) or the index number (int)
1799          of the phase in the project, numbered starting from 0.
1800          Omitting the parameter or the string 'all' indicates that parameters in
1801          all phases should be set.
1802
1803        Note that refinement parameters are categorized as one of three types:
1804
1805        1. Histogram parameters
1806        2. Phase parameters
1807        3. Histogram-and-Phase (HAP) parameters
1808       
1809        .. seealso::
1810            :meth:`G2PwdrData.set_refinements`
1811            :meth:`G2PwdrData.clear_refinements`
1812            :meth:`G2Phase.set_refinements`
1813            :meth:`G2Phase.clear_refinements`
1814            :meth:`G2Phase.set_HAP_refinements`
1815            :meth:`G2Phase.clear_HAP_refinements`"""
1816
1817        if histogram == 'all':
1818            hists = self.histograms()
1819        elif isinstance(histogram, list) or isinstance(histogram, tuple):
1820            hists = []
1821            for h in histogram:
1822                if isinstance(h, str) or isinstance(h, int):
1823                    hists.append(self.histogram(h))
1824                else:
1825                    hists.append(h)
1826        elif isinstance(histogram, str) or isinstance(histogram, int):
1827            hists = [self.histogram(histogram)]
1828        else:
1829            hists = [histogram]
1830
1831        if phase == 'all':
1832            phases = self.phases()
1833        elif isinstance(phase, list) or isinstance(phase, tuple):
1834            phases = []
1835            for ph in phase:
1836                if isinstance(ph, str) or isinstance(ph, int):
1837                    phases.append(self.phase(ph))
1838                else:
1839                    phases.append(ph)
1840        elif isinstance(phase, str) or isinstance(phase, int):
1841            phases = [self.phase(phase)]
1842        else:
1843            phases = [phase]
1844
1845        # TODO: HAP parameters:
1846        #   Babinet
1847        #   Extinction
1848        #   HStrain
1849        #   Mustrain
1850        #   Pref. Ori
1851        #   Size
1852
1853        pwdr_set = {}
1854        phase_set = {}
1855        hap_set = {}
1856        for key, val in refinement.get('set', {}).items():
1857            # Apply refinement options
1858            if G2PwdrData.is_valid_refinement_key(key):
1859                pwdr_set[key] = val
1860            elif G2Phase.is_valid_refinement_key(key):
1861                phase_set[key] = val
1862            elif G2Phase.is_valid_HAP_refinement_key(key):
1863                hap_set[key] = val
1864            else:
1865                raise ValueError("Unknown refinement key", key)
1866
1867        for hist in hists:
1868            hist.set_refinements(pwdr_set)
1869        for phase in phases:
1870            phase.set_refinements(phase_set)
1871        for phase in phases:
1872            phase.set_HAP_refinements(hap_set, hists)
1873
1874        pwdr_clear = {}
1875        phase_clear = {}
1876        hap_clear = {}
1877        for key, val in refinement.get('clear', {}).items():
1878            # Clear refinement options
1879            if G2PwdrData.is_valid_refinement_key(key):
1880                pwdr_clear[key] = val
1881            elif G2Phase.is_valid_refinement_key(key):
1882                phase_clear[key] = val
1883            elif G2Phase.is_valid_HAP_refinement_key(key):
1884                hap_set[key] = val
1885            else:
1886                raise ValueError("Unknown refinement key", key)
1887
1888        for hist in hists:
1889            hist.clear_refinements(pwdr_clear)
1890        for phase in phases:
1891            phase.clear_refinements(phase_clear)
1892        for phase in phases:
1893            phase.clear_HAP_refinements(hap_clear, hists)
1894
1895    def index_ids(self):
1896        import GSASIIstrIO as G2strIO
1897        self.save()
1898        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
1899
1900    def add_constraint_raw(self, cons_scope, constr):
1901        """Adds a constraint of type consType to the project.
1902        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
1903
1904        WARNING it does not check the constraint is well-constructed"""
1905        constrs = self.data['Constraints']['data']
1906        if 'Global' not in constrs:
1907            constrs['Global'] = []
1908        constrs[cons_scope].append(constr)
1909
1910    def hold_many(self, vars, type):
1911        """Apply holds for all the variables in vars, for constraint of a given type.
1912
1913        type is passed directly to add_constraint_raw as consType
1914
1915        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
1916            string variable specifiers, or arguments for :meth:`make_var_obj`
1917        :param str type: A string constraint type specifier. See
1918            :class:`G2Project.add_constraint_raw`
1919
1920        """
1921        for var in vars:
1922            if isinstance(var, str):
1923                var = self.make_var_obj(var)
1924            elif not isinstance(var, G2obj.G2VarObj):
1925                var = self.make_var_obj(*var)
1926            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
1927
1928    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
1929                     reloadIdx=True):
1930        """Wrapper to create a G2VarObj. Takes either a string representaiton ("p:h:name:a")
1931        or individual names of phase, histogram, varname, and atomId.
1932
1933        Automatically converts string phase, hist, or atom names into the ID required
1934        by G2VarObj."""
1935
1936        if reloadIdx:
1937            self.index_ids()
1938
1939        # If string representation, short circuit
1940        if hist is None and varname is None and atomId is None:
1941            if isinstance(phase, str) and ':' in phase:
1942                return G2obj.G2VarObj(phase)
1943
1944        # Get phase index
1945        phaseObj = None
1946        if isinstance(phase, G2Phase):
1947            phaseObj = phase
1948            phase = G2obj.PhaseRanIdLookup[phase.ranId]
1949        elif phase in self.data['Phases']:
1950            phaseObj = self.phase(phase)
1951            phaseRanId = phaseObj.ranId
1952            phase = G2obj.PhaseRanIdLookup[phaseRanId]
1953
1954        # Get histogram index
1955        if isinstance(hist, G2PwdrData):
1956            hist = G2obj.HistRanIdLookup[hist.ranId]
1957        elif hist in self.data:
1958            histRanId = self.histogram(hist).ranId
1959            hist = G2obj.HistRanIdLookup[histRanId]
1960
1961        # Get atom index (if any)
1962        if isinstance(atomId, G2AtomRecord):
1963            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
1964        elif phaseObj:
1965            atomObj = phaseObj.atom(atomId)
1966            if atomObj:
1967                atomRanId = atomObj.ranId
1968                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
1969
1970        return G2obj.G2VarObj(phase, hist, varname, atomId)
1971
1972
1973class G2AtomRecord(G2ObjectWrapper):
1974    """Wrapper for an atom record. Has convenient accessors via @property.
1975
1976
1977    Available accessors: label, type, refinement_flags, coordinates,
1978        occupancy, ranId, id, adp_flag, uiso
1979
1980    Example:
1981
1982    >>> atom = some_phase.atom("O3")
1983    >>> # We can access the underlying data format
1984    >>> atom.data
1985    ['O3', 'O-2', '', ... ]
1986    >>> # We can also use wrapper accessors
1987    >>> atom.coordinates
1988    (0.33, 0.15, 0.5)
1989    >>> atom.refinement_flags
1990    u'FX'
1991    >>> atom.ranId
1992    4615973324315876477
1993    >>> atom.occupancy
1994    1.0
1995    """
1996    def __init__(self, data, indices, proj):
1997        self.data = data
1998        self.cx, self.ct, self.cs, self.cia = indices
1999        self.proj = proj
2000
2001    @property
2002    def label(self):
2003        return self.data[self.ct-1]
2004
2005    @property
2006    def type(self):
2007        return self.data[self.ct]
2008
2009    @property
2010    def refinement_flags(self):
2011        return self.data[self.ct+1]
2012
2013    @refinement_flags.setter
2014    def refinement_flags(self, other):
2015        # Automatically check it is a valid refinement
2016        for c in other:
2017            if c not in ' FXU':
2018                raise ValueError("Invalid atom refinement: ", other)
2019        self.data[self.ct+1] = other
2020
2021    @property
2022    def coordinates(self):
2023        return tuple(self.data[self.cx:self.cx+3])
2024
2025    @property
2026    def occupancy(self):
2027        return self.data[self.cx+3]
2028
2029    @occupancy.setter
2030    def occupancy(self, val):
2031        self.data[self.cx+3] = float(val)
2032
2033    @property
2034    def ranId(self):
2035        return self.data[self.cia+8]
2036
2037    @property
2038    def adp_flag(self):
2039        # Either 'I' or 'A'
2040        return self.data[self.cia]
2041
2042    @property
2043    def uiso(self):
2044        if self.adp_flag == 'I':
2045            return self.data[self.cia+1]
2046        else:
2047            return self.data[self.cia+2:self.cia+8]
2048
2049    @uiso.setter
2050    def uiso(self, value):
2051        if self.adp_flag == 'I':
2052            self.data[self.cia+1] = float(value)
2053        else:
2054            assert len(value) == 6
2055            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2056
2057
2058class G2PwdrData(G2ObjectWrapper):
2059    """Wraps a Powder Data Histogram."""
2060    def __init__(self, data, proj):
2061        self.data = data
2062        self.proj = proj
2063
2064    @staticmethod
2065    def is_valid_refinement_key(key):
2066        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2067                      'Instrument Parameters']
2068        return key in valid_keys
2069
2070    @property
2071    def name(self):
2072        return self.data['data'][-1]
2073
2074    @property
2075    def ranId(self):
2076        return self.data['data'][0]['ranId']
2077
2078    @property
2079    def residuals(self):
2080        '''Provides a dictionary with with the R-factors for this histogram.
2081        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2082        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2083        '''
2084        data = self.data['data'][0]
2085        return {key: data[key] for key in data
2086                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2087                   or ':' in key}
2088
2089    @property
2090    def InstrumentParameters(self):
2091        '''Provides a dictionary with with the Instrument Parameters
2092        for this histogram.
2093        '''
2094        return self.data['Instrument Parameters'][0]
2095
2096    @property
2097    def SampleParameters(self):
2098        '''Provides a dictionary with with the Sample Parameters
2099        for this histogram.
2100        '''
2101        return self.data['Sample Parameters']
2102   
2103    @property
2104    def id(self):
2105        self.proj.update_ids()
2106        return self.data['data'][0]['hId']
2107
2108    @id.setter
2109    def id(self, val):
2110        self.data['data'][0]['hId'] = val
2111
2112    def fit_fixed_points(self):
2113        """Attempts to apply a background fit to the fixed points currently specified."""
2114        def SetInstParms(Inst):
2115            dataType = Inst['Type'][0]
2116            insVary = []
2117            insNames = []
2118            insVals = []
2119            for parm in Inst:
2120                insNames.append(parm)
2121                insVals.append(Inst[parm][1])
2122                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2123                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2124                        Inst[parm][2] = False
2125            instDict = dict(zip(insNames, insVals))
2126            instDict['X'] = max(instDict['X'], 0.01)
2127            instDict['Y'] = max(instDict['Y'], 0.01)
2128            if 'SH/L' in instDict:
2129                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2130            return dataType, instDict, insVary
2131
2132        bgrnd = self.data['Background']
2133
2134        # Need our fixed points in order
2135        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2136        X = [x for x, y in bgrnd[1]['FixedPoints']]
2137        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2138
2139        limits = self.data['Limits'][1]
2140        if X[0] > limits[0]:
2141            X = [limits[0]] + X
2142            Y = [Y[0]] + Y
2143        if X[-1] < limits[1]:
2144            X += [limits[1]]
2145            Y += [Y[-1]]
2146
2147        # Some simple lookups
2148        controls = self.proj['Controls']['data']
2149        inst, inst2 = self.data['Instrument Parameters']
2150        pwddata = self.data['data'][1]
2151
2152        # Construct the data for background fitting
2153        xBeg = np.searchsorted(pwddata[0], limits[0])
2154        xFin = np.searchsorted(pwddata[0], limits[1])
2155        xdata = pwddata[0][xBeg:xFin]
2156        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2157
2158        W = [1]*len(xdata)
2159        Z = [0]*len(xdata)
2160
2161        dataType, insDict, insVary = SetInstParms(inst)
2162        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2163
2164        # Do the fit
2165        data = np.array([xdata, ydata, W, Z, Z, Z])
2166        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2167                        prevVaryList=bakVary, controls=controls)
2168
2169        # Post-fit
2170        parmDict = {}
2171        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2172        parmDict.update(bakDict)
2173        parmDict.update(insDict)
2174        pwddata[3][xBeg:xFin] *= 0
2175        pwddata[5][xBeg:xFin] *= 0
2176        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2177
2178        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2179        # TODO update background
2180        self.proj.save()
2181
2182    def getdata(self,datatype):
2183        '''Provides access to the histogram data of the selected data type
2184
2185        :param str datatype: must be one of the following values (case is ignored)
2186       
2187           * 'X': the 2theta or TOF values for the pattern
2188           * 'Yobs': the observed intensity values
2189           * 'Yweight': the weights for each data point (1/sigma**2)
2190           * 'Ycalc': the computed intensity values
2191           * 'Background': the computed background values
2192           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2193
2194        :returns: an numpy MaskedArray with data values of the requested type
2195       
2196        '''
2197        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2198        if datatype.lower() not in enums:
2199            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2200        return self.data['data'][1][enums.index(datatype.lower())]
2201       
2202    def y_calc(self):
2203        return self.data['data'][1][3]
2204
2205    def Export(self,fileroot,extension):
2206        '''Write the histogram into a file. The path is specified by fileroot and
2207        extension.
2208       
2209        :param str fileroot: name of the file, optionally with a path (extension is
2210           ignored)
2211        :param str extension: includes '.', must match an extension in global
2212           exportersByExtension['powder'] or a Exception is raised.
2213        :returns: name of file that was written
2214        '''
2215        if extension not in exportersByExtension.get('powder',[]):
2216            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2217        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2218        obj = exportersByExtension['powder'][extension]
2219        obj.SetFromArray(hist=self.data,histname=self.name)
2220        obj.Writer(self.name,fil)
2221           
2222    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2223        try:
2224            import matplotlib.pyplot as plt
2225            data = self.data['data'][1]
2226            if Yobs:
2227                plt.plot(data[0], data[1], label='Yobs')
2228            if Ycalc:
2229                plt.plot(data[0], data[3], label='Ycalc')
2230            if Background:
2231                plt.plot(data[0], data[4], label='Background')
2232            if Residual:
2233                plt.plot(data[0], data[5], label="Residual")
2234        except ImportError:
2235            pass
2236
2237    def get_wR(self):
2238        """returns the overall weighted profile R factor for a histogram
2239       
2240        :returns: a wR value as a percentage or None if not defined
2241        """
2242        return self['data'][0].get('wR')
2243
2244    def set_refinements(self, refs):
2245        """Sets the refinement parameter 'key' to the specification 'value'
2246
2247        :param dict refs: A dictionary of the parameters to be set. See
2248                          :ref:`Histogram_parameters_table` for a description of
2249                          what these dictionaries should be.
2250
2251        :returns: None
2252
2253        """
2254        do_fit_fixed_points = False
2255        for key, value in refs.items():
2256            if key == 'Limits':
2257                old_limits = self.data['Limits'][1]
2258                new_limits = value
2259                if isinstance(new_limits, dict):
2260                    if 'low' in new_limits:
2261                        old_limits[0] = new_limits['low']
2262                    if 'high' in new_limits:
2263                        old_limits[1] = new_limits['high']
2264                else:
2265                    old_limits[0], old_limits[1] = new_limits
2266            elif key == 'Sample Parameters':
2267                sample = self.data['Sample Parameters']
2268                for sparam in value:
2269                    if sparam not in sample:
2270                        raise ValueError("Unknown refinement parameter, "
2271                                         + str(sparam))
2272                    sample[sparam][1] = True
2273            elif key == 'Background':
2274                bkg, peaks = self.data['Background']
2275
2276                # If True or False, just set the refine parameter
2277                if value in (True, False):
2278                    bkg[1] = value
2279                    return
2280
2281                if 'type' in value:
2282                    bkg[0] = value['type']
2283                if 'refine' in value:
2284                    bkg[1] = value['refine']
2285                if 'no. coeffs' in value:
2286                    cur_coeffs = bkg[2]
2287                    n_coeffs = value['no. coeffs']
2288                    if n_coeffs > cur_coeffs:
2289                        for x in range(n_coeffs - cur_coeffs):
2290                            bkg.append(0.0)
2291                    else:
2292                        for _ in range(cur_coeffs - n_coeffs):
2293                            bkg.pop()
2294                    bkg[2] = n_coeffs
2295                if 'coeffs' in value:
2296                    bkg[3:] = value['coeffs']
2297                if 'FixedPoints' in value:
2298                    peaks['FixedPoints'] = [(float(a), float(b))
2299                                            for a, b in value['FixedPoints']]
2300                if value.get('fit fixed points', False):
2301                    do_fit_fixed_points = True
2302
2303            elif key == 'Instrument Parameters':
2304                instrument, secondary = self.data['Instrument Parameters']
2305                for iparam in value:
2306                    try:
2307                        instrument[iparam][2] = True
2308                    except IndexError:
2309                        raise ValueError("Invalid key:", iparam)
2310            else:
2311                raise ValueError("Unknown key:", key)
2312        # Fit fixed points after the fact - ensure they are after fixed points
2313        # are added
2314        if do_fit_fixed_points:
2315            # Background won't be fit if refinement flag not set
2316            orig = self.data['Background'][0][1]
2317            self.data['Background'][0][1] = True
2318            self.fit_fixed_points()
2319            # Restore the previous value
2320            self.data['Background'][0][1] = orig
2321
2322    def clear_refinements(self, refs):
2323        """Clears the refinement parameter 'key' and its associated value.
2324
2325        :param dict refs: A dictionary of parameters to clear."""
2326        for key, value in refs.items():
2327            if key == 'Limits':
2328                old_limits, cur_limits = self.data['Limits']
2329                cur_limits[0], cur_limits[1] = old_limits
2330            elif key == 'Sample Parameters':
2331                sample = self.data['Sample Parameters']
2332                for sparam in value:
2333                    sample[sparam][1] = False
2334            elif key == 'Background':
2335                bkg, peaks = self.data['Background']
2336
2337                # If True or False, just set the refine parameter
2338                if value in (True, False):
2339                    bkg[1] = False
2340                    return
2341
2342                bkg[1] = False
2343                if 'FixedPoints' in value:
2344                    if 'FixedPoints' in peaks:
2345                        del peaks['FixedPoints']
2346            elif key == 'Instrument Parameters':
2347                instrument, secondary = self.data['Instrument Parameters']
2348                for iparam in value:
2349                    instrument[iparam][2] = False
2350            else:
2351                raise ValueError("Unknown key:", key)
2352
2353    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2354        '''Adds a single peak to the peak list
2355        :param float area: peak area
2356        :param float dspace: peak position as d-space (A)
2357        :param float Q: peak position as Q (A-1)
2358        :param float ttheta: peak position as 2Theta (deg)
2359
2360        Note: only one of the parameters dspace, Q or ttheta may be specified
2361        '''
2362        import GSASIIlattice as G2lat
2363        import GSASIImath as G2mth
2364        if (not dspace) + (not Q) + (not ttheta) != 2:
2365            print('add_peak error: too many or no peak position(s) specified')
2366            return
2367        pos = ttheta
2368        Parms,Parms2 = self.data['Instrument Parameters']
2369        if Q:
2370            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2371        elif dspace:
2372            pos = G2lat.Dsp2pos(Parms,dspace)
2373        peaks = self.data['Peak List']
2374        peaks['sigDict'] = {}        #no longer valid
2375        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2376
2377    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2378        '''Set refinement flags for peaks
2379       
2380        :param list peaklist: a list of peaks to change flags. If None (default), changes
2381          are made to all peaks.
2382        :param bool area: Sets or clears the refinement flag for the peak area value.
2383          If None (the default), no change is made.
2384        :param bool pos: Sets or clears the refinement flag for the peak position value.
2385          If None (the default), no change is made.
2386        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2387          If None (the default), no change is made.
2388        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2389          If None (the default), no change is made.
2390         
2391        Note that when peaks are first created the area flag is on and the other flags are
2392        initially off.
2393
2394        Example::
2395       
2396           set_peakFlags(sig=False,gam=True)
2397
2398        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
2399        all peaks. The position and area flags are not changed from their previous values.
2400        '''
2401        peaks = self.data['Peak List']
2402        if peaklist is None:
2403            peaklist = range(len(peaks['peaks']))
2404        for i in peaklist:
2405            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
2406                if var is not None:
2407                    peaks['peaks'][i][j] = var
2408           
2409    def refine_peaks(self):
2410        '''Causes a refinement of peak position, background and instrument parameters
2411        '''
2412        import GSASIIpwd as G2pwd
2413        controls = self.proj.data.get('Controls',{})
2414        controls = controls.get('data',
2415                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
2416                            )
2417        peaks = self.data['Peak List']
2418        Parms,Parms2 = self.data['Instrument Parameters']
2419        background = self.data['Background']
2420        limits = self.data['Limits'][1]
2421        bxye = np.zeros(len(self.data['data'][1][1]))
2422        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
2423                                           Parms,Parms2,self.data['data'][1],bxye,[],
2424                                           False,controls,None)[0]
2425
2426    def SaveProfile(self,filename):
2427        '''Writes a GSAS-II (new style) .instprm file
2428        '''
2429        data,Parms2 = self.data['Instrument Parameters']
2430        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2431        File = open(filename,'w')
2432        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
2433        for item in data:
2434            File.write(item+':'+str(data[item][1])+'\n')
2435        File.close()
2436        print ('Instrument parameters saved to: '+filename)
2437
2438    def LoadProfile(self,filename):
2439        '''Reads a GSAS-II (new style) .instprm file and overwrites the current parameters
2440        '''
2441        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2442        File = open(filename,'r')
2443        S = File.readline()
2444        newItems = []
2445        newVals = []
2446        Found = False
2447        while S:
2448            if S[0] == '#':
2449                if Found:
2450                    break
2451                if 'Bank' in S:
2452                    if bank == int(S.split(':')[0].split()[1]):
2453                        S = File.readline()
2454                        continue
2455                    else:
2456                        S = File.readline()
2457                        while S and '#Bank' not in S:
2458                            S = File.readline()
2459                        continue
2460                else:   #a non #Bank file
2461                    S = File.readline()
2462                    continue
2463            Found = True
2464            [item,val] = S[:-1].split(':')
2465            newItems.append(item)
2466            try:
2467                newVals.append(float(val))
2468            except ValueError:
2469                newVals.append(val)                       
2470            S = File.readline()               
2471        File.close()
2472        LoadG2fil()
2473        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
2474
2475       
2476class G2Phase(G2ObjectWrapper):
2477    """A wrapper object around a given phase.
2478
2479    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
2480    """
2481    def __init__(self, data, name, proj):
2482        self.data = data
2483        self.name = name
2484        self.proj = proj
2485
2486    @staticmethod
2487    def is_valid_refinement_key(key):
2488        valid_keys = ["Cell", "Atoms", "LeBail"]
2489        return key in valid_keys
2490
2491    @staticmethod
2492    def is_valid_HAP_refinement_key(key):
2493        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
2494                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
2495        return key in valid_keys
2496
2497    def atom(self, atomlabel):
2498        """Returns the atom specified by atomlabel, or None if it does not
2499        exist.
2500
2501        :param str atomlabel: The name of the atom (e.g. "O2")
2502        :returns: A :class:`G2AtomRecord` object
2503            representing the atom.
2504        """
2505        # Consult GSASIIobj.py for the meaning of this
2506        cx, ct, cs, cia = self.data['General']['AtomPtrs']
2507        ptrs = [cx, ct, cs, cia]
2508        atoms = self.data['Atoms']
2509        for atom in atoms:
2510            if atom[ct-1] == atomlabel:
2511                return G2AtomRecord(atom, ptrs, self.proj)
2512
2513    def atoms(self):
2514        """Returns a list of atoms present in the phase.
2515
2516        :returns: A list of :class:`G2AtomRecord` objects.
2517
2518        .. seealso::
2519            :meth:`G2Phase.atom`
2520            :class:`G2AtomRecord`
2521        """
2522        ptrs = self.data['General']['AtomPtrs']
2523        output = []
2524        atoms = self.data['Atoms']
2525        for atom in atoms:
2526            output.append(G2AtomRecord(atom, ptrs, self.proj))
2527        return output
2528
2529    def histograms(self):
2530        output = []
2531        for hname in self.data.get('Histograms', {}).keys():
2532            output.append(self.proj.histogram(hname))
2533        return output
2534
2535    @property
2536    def ranId(self):
2537        return self.data['ranId']
2538
2539    @property
2540    def id(self):
2541        return self.data['pId']
2542
2543    @id.setter
2544    def id(self, val):
2545        self.data['pId'] = val
2546
2547    def get_cell(self):
2548        """Returns a dictionary of the cell parameters, with keys:
2549            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
2550
2551        :returns: a dict
2552
2553        .. seealso::
2554           :meth:`G2Phase.get_cell_and_esd`
2555
2556        """
2557        cell = self.data['General']['Cell']
2558        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
2559                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
2560                'volume': cell[7]}
2561
2562    def get_cell_and_esd(self):
2563        """
2564        Returns a pair of dictionaries, the first representing the unit cell, the second
2565        representing the estimated standard deviations of the unit cell.
2566
2567        :returns: a tuple of two dictionaries
2568
2569        .. seealso::
2570           :meth:`G2Phase.get_cell`
2571
2572        """
2573        # translated from GSASIIstrIO.ExportBaseclass.GetCell
2574        import GSASIIstrIO as G2stIO
2575        import GSASIIlattice as G2lat
2576        import GSASIImapvars as G2mv
2577        try:
2578            pfx = str(self.id) + '::'
2579            sgdata = self['General']['SGData']
2580            covDict = self.proj['Covariance']['data']
2581
2582            parmDict = dict(zip(covDict.get('varyList',[]),
2583                                covDict.get('variables',[])))
2584            sigDict = dict(zip(covDict.get('varyList',[]),
2585                               covDict.get('sig',[])))
2586
2587            if covDict.get('covMatrix') is not None:
2588                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2589                                                  covDict['varyList'],
2590                                                  parmDict))
2591
2592            A, sigA = G2stIO.cellFill(pfx, sgdata, parmDict, sigDict)
2593            cellSig = G2stIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
2594            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
2595            cellDict, cellSigDict = {}, {}
2596            for i, key in enumerate(['length_a', 'length_b', 'length_c',
2597                                     'angle_alpha', 'angle_beta', 'angle_gamma',
2598                                     'volume']):
2599                cellDict[key] = cellList[i]
2600                cellSigDict[key] = cellSig[i]
2601            return cellDict, cellSigDict
2602        except KeyError:
2603            cell = self.get_cell()
2604            return cell, {key: 0.0 for key in cell}
2605
2606    def export_CIF(self, outputname, quickmode=True):
2607        """Write this phase to a .cif file named outputname
2608
2609        :param str outputname: The name of the .cif file to write to
2610        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
2611        # This code is all taken from exports/G2export_CIF.py
2612        # Functions copied have the same names
2613        import GSASIImath as G2mth
2614        import GSASIImapvars as G2mv
2615        from exports import G2export_CIF as cif
2616
2617        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
2618        CIFname = os.path.splitext(self.proj.filename)[0]
2619        CIFname = os.path.split(CIFname)[1]
2620        CIFname = ''.join([c if ord(c) < 128 else ''
2621                           for c in CIFname.replace(' ', '_')])
2622        try:
2623            author = self.proj['Controls']['data'].get('Author','').strip()
2624        except KeyError:
2625            pass
2626        oneblock = True
2627
2628        covDict = self.proj['Covariance']['data']
2629        parmDict = dict(zip(covDict.get('varyList',[]),
2630                            covDict.get('variables',[])))
2631        sigDict = dict(zip(covDict.get('varyList',[]),
2632                           covDict.get('sig',[])))
2633
2634        if covDict.get('covMatrix') is not None:
2635            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2636                                              covDict['varyList'],
2637                                              parmDict))
2638
2639        with open(outputname, 'w') as fp:
2640            fp.write(' \n' + 70*'#' + '\n')
2641            cif.WriteCIFitem(fp, 'data_' + CIFname)
2642            # from exports.G2export_CIF.WritePhaseInfo
2643            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
2644            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
2645            # TODO get esds
2646            cellDict = self.get_cell()
2647            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
2648            names = ['length_a','length_b','length_c',
2649                     'angle_alpha','angle_beta ','angle_gamma',
2650                     'volume']
2651            for key, val in cellDict.items():
2652                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
2653
2654            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
2655                         self.data['General']['SGData']['SGSys'])
2656
2657            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
2658            # regularize capitalization and remove trailing H/R
2659            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
2660            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
2661
2662            # generate symmetry operations including centering and center of symmetry
2663            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
2664                self.data['General']['SGData'])
2665            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
2666            for i, op in enumerate(SymOpList,start=1):
2667                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
2668
2669            # TODO skipped histograms, exports/G2export_CIF.py:880
2670
2671            # report atom params
2672            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
2673                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
2674                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
2675            else:
2676                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
2677            # report cell contents
2678            cif.WriteComposition(fp, self.data, self.name, parmDict)
2679            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
2680                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
2681                raise NotImplementedError("only quickmode currently supported")
2682            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
2683                cif.WriteCIFitem(fp,'\n# Difference density results')
2684                MinMax = self.data['General']['Map']['minmax']
2685                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
2686                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
2687
2688
2689    def set_refinements(self, refs):
2690        """Sets the refinement parameter 'key' to the specification 'value'
2691
2692        :param dict refs: A dictionary of the parameters to be set. See
2693                          :ref:`Phase_parameters_table` for a description of
2694                          this dictionary.
2695
2696        :returns: None"""
2697        for key, value in refs.items():
2698            if key == "Cell":
2699                self.data['General']['Cell'][0] = value
2700
2701            elif key == "Atoms":
2702                for atomlabel, atomrefinement in value.items():
2703                    if atomlabel == 'all':
2704                        for atom in self.atoms():
2705                            atom.refinement_flags = atomrefinement
2706                    else:
2707                        atom = self.atom(atomlabel)
2708                        if atom is None:
2709                            raise ValueError("No such atom: " + atomlabel)
2710                        atom.refinement_flags = atomrefinement
2711
2712            elif key == "LeBail":
2713                hists = self.data['Histograms']
2714                for hname, hoptions in hists.items():
2715                    if 'LeBail' not in hoptions:
2716                        hoptions['newLeBail'] = bool(True)
2717                    hoptions['LeBail'] = bool(value)
2718            else:
2719                raise ValueError("Unknown key:", key)
2720
2721    def clear_refinements(self, refs):
2722        """Clears a given set of parameters.
2723
2724        :param dict refs: The parameters to clear"""
2725        for key, value in refs.items():
2726            if key == "Cell":
2727                self.data['General']['Cell'][0] = False
2728            elif key == "Atoms":
2729                cx, ct, cs, cia = self.data['General']['AtomPtrs']
2730
2731                for atomlabel in value:
2732                    atom = self.atom(atomlabel)
2733                    # Set refinement to none
2734                    atom.refinement_flags = ' '
2735            elif key == "LeBail":
2736                hists = self.data['Histograms']
2737                for hname, hoptions in hists.items():
2738                    if 'LeBail' not in hoptions:
2739                        hoptions['newLeBail'] = True
2740                    hoptions['LeBail'] = False
2741            else:
2742                raise ValueError("Unknown key:", key)
2743
2744    def set_HAP_refinements(self, refs, histograms='all'):
2745        """Sets the given HAP refinement parameters between this phase and
2746        the given histograms
2747
2748        :param dict refs: A dictionary of the parameters to be set. See
2749                          :ref:`HAP_parameters_table` for a description of this
2750                          dictionary.
2751        :param histograms: Either 'all' (default) or a list of the histograms
2752            whose HAP parameters will be set with this phase. Histogram and phase
2753            must already be associated
2754
2755        :returns: None
2756        """
2757        if histograms == 'all':
2758            histograms = self.data['Histograms'].values()
2759        else:
2760            histograms = [self.data['Histograms'][h.name] for h in histograms]
2761
2762        for key, val in refs.items():
2763            for h in histograms:
2764                if key == 'Babinet':
2765                    try:
2766                        sets = list(val)
2767                    except ValueError:
2768                        sets = ['BabA', 'BabU']
2769                    for param in sets:
2770                        if param not in ['BabA', 'BabU']:
2771                            raise ValueError("Not sure what to do with" + param)
2772                        for hist in histograms:
2773                            hist['Babinet'][param][1] = True
2774                elif key == 'Extinction':
2775                    for h in histograms:
2776                        h['Extinction'][1] = bool(val)
2777                elif key == 'HStrain':
2778                    for h in histograms:
2779                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
2780                elif key == 'Mustrain':
2781                    for h in histograms:
2782                        mustrain = h['Mustrain']
2783                        newType = None
2784                        direction = None
2785                        if isinstance(val, strtypes):
2786                            if val in ['isotropic', 'uniaxial', 'generalized']:
2787                                newType = val
2788                            else:
2789                                raise ValueError("Not a Mustrain type: " + val)
2790                        elif isinstance(val, dict):
2791                            newType = val.get('type', None)
2792                            direction = val.get('direction', None)
2793
2794                        if newType:
2795                            mustrain[0] = newType
2796                            if newType == 'isotropic':
2797                                mustrain[2][0] = True == val.get('refine',False)
2798                                mustrain[5] = [False for p in mustrain[4]]
2799                            elif newType == 'uniaxial':
2800                                if 'refine' in val:
2801                                    mustrain[2][0] = False
2802                                    types = val['refine']
2803                                    if isinstance(types, strtypes):
2804                                        types = [types]
2805                                    elif isinstance(types, bool):
2806                                        mustrain[2][1] = types
2807                                        mustrain[2][2] = types
2808                                        types = []
2809                                    else:
2810                                        raise ValueError("Not sure what to do with: "
2811                                                         + str(types))
2812                                else:
2813                                    types = []
2814
2815                                for unitype in types:
2816                                    if unitype == 'equatorial':
2817                                        mustrain[2][0] = True
2818                                    elif unitype == 'axial':
2819                                        mustrain[2][1] = True
2820                                    else:
2821                                        msg = 'Invalid uniaxial mustrain type'
2822                                        raise ValueError(msg + ': ' + unitype)
2823                            else:  # newtype == 'generalized'
2824                                mustrain[2] = [False for p in mustrain[1]]
2825                                if 'refine' in val:
2826                                    mustrain[5] = [True == val['refine']]*len(mustrain[5])
2827
2828                        if direction:
2829                            if len(direction) != 3:
2830                                raise ValueError("Expected hkl, found", direction)
2831                            direction = [int(n) for n in direction]
2832                            mustrain[3] = direction
2833                elif key == 'Size':
2834                    newSize = None
2835                    if 'value' in val:
2836                        newSize = float(val['value'])
2837                    for h in histograms:
2838                        size = h['Size']
2839                        newType = None
2840                        direction = None
2841                        if isinstance(val, strtypes):
2842                            if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
2843                                newType = val
2844                            else:
2845                                raise ValueError("Not a valid Size type: " + val)
2846                        elif isinstance(val, dict):
2847                            newType = val.get('type', None)
2848                            direction = val.get('direction', None)
2849
2850                        if newType:
2851                            size[0] = newType
2852                            refine = True == val.get('refine')
2853                            if newType == 'isotropic' and refine is not None:
2854                                size[2][0] = bool(refine)
2855                                if newSize: size[1][0] = newSize
2856                            elif newType == 'uniaxial' and refine is not None:
2857                                size[2][1] = bool(refine)
2858                                size[2][2] = bool(refine)
2859                                if newSize: size[1][1] = size[1][2] =newSize
2860                            elif newType == 'ellipsoidal' and refine is not None:
2861                                size[5] = [bool(refine) for p in size[5]]
2862                                if newSize: size[4] = [newSize for p in size[4]]
2863
2864                        if direction:
2865                            if len(direction) != 3:
2866                                raise ValueError("Expected hkl, found", direction)
2867                            direction = [int(n) for n in direction]
2868                            size[3] = direction
2869                elif key == 'Pref.Ori.':
2870                    for h in histograms:
2871                        h['Pref.Ori.'][2] = bool(val)
2872                elif key == 'Show':
2873                    for h in histograms:
2874                        h['Show'] = bool(val)
2875                elif key == 'Use':
2876                    for h in histograms:
2877                        h['Use'] = bool(val)
2878                elif key == 'Scale':
2879                    for h in histograms:
2880                        h['Scale'][1] = bool(val)
2881                else:
2882                    print(u'Unknown HAP key: '+key)
2883
2884    def getHAPvalues(self, histname):
2885        """Returns a dict with HAP values for the selected histogram
2886
2887        :param histogram: is a histogram object (:class:`G2PwdrData`) or
2888            a histogram name or the index number of the histogram
2889
2890        :returns: HAP value dict
2891        """
2892        if isinstance(histname, G2PwdrData):
2893            histname = histname.name
2894        elif histname in self.data['Histograms']:
2895            pass
2896        elif type(histname) is int:
2897            histname = self.proj.histograms()[histname].name
2898        else:
2899            raise G2ScriptException("Invalid histogram reference: "+str(histname))
2900        return self.data['Histograms'][histname]
2901                   
2902    def clear_HAP_refinements(self, refs, histograms='all'):
2903        """Clears the given HAP refinement parameters between this phase and
2904        the given histograms
2905
2906        :param dict refs: A dictionary of the parameters to be cleared.
2907        :param histograms: Either 'all' (default) or a list of the histograms
2908            whose HAP parameters will be cleared with this phase. Histogram and
2909            phase must already be associated
2910
2911        :returns: None
2912        """
2913        if histograms == 'all':
2914            histograms = self.data['Histograms'].values()
2915        else:
2916            histograms = [self.data['Histograms'][h.name] for h in histograms]
2917
2918        for key, val in refs.items():
2919            for h in histograms:
2920                if key == 'Babinet':
2921                    try:
2922                        sets = list(val)
2923                    except ValueError:
2924                        sets = ['BabA', 'BabU']
2925                    for param in sets:
2926                        if param not in ['BabA', 'BabU']:
2927                            raise ValueError("Not sure what to do with" + param)
2928                        for hist in histograms:
2929                            hist['Babinet'][param][1] = False
2930                elif key == 'Extinction':
2931                    for h in histograms:
2932                        h['Extinction'][1] = False
2933                elif key == 'HStrain':
2934                    for h in histograms:
2935                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
2936                elif key == 'Mustrain':
2937                    for h in histograms:
2938                        mustrain = h['Mustrain']
2939                        mustrain[2] = [False for p in mustrain[2]]
2940                        mustrain[5] = [False for p in mustrain[4]]
2941                elif key == 'Pref.Ori.':
2942                    for h in histograms:
2943                        h['Pref.Ori.'][2] = False
2944                elif key == 'Show':
2945                    for h in histograms:
2946                        h['Show'] = False
2947                elif key == 'Size':
2948                    for h in histograms:
2949                        size = h['Size']
2950                        size[2] = [False for p in size[2]]
2951                        size[5] = [False for p in size[5]]
2952                elif key == 'Use':
2953                    for h in histograms:
2954                        h['Use'] = False
2955                elif key == 'Scale':
2956                    for h in histograms:
2957                        h['Scale'][1] = False
2958                else:
2959                    print(u'Unknown HAP key: '+key)
2960
2961
2962##########################
2963# Command Line Interface #
2964##########################
2965# Each of these takes an argparse.Namespace object as their argument,
2966# representing the parsed command-line arguments for the relevant subcommand.
2967# The argument specification for each is in the subcommands dictionary (see
2968# below)
2969
2970commandhelp={}
2971commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
2972def create(args):
2973    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
2974
2975  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2976                                  [-i IPARAMS [IPARAMS ...]]
2977                                  [-p PHASES [PHASES ...]]
2978                                  filename
2979                                 
2980positional arguments::
2981
2982  filename              the project file to create. should end in .gpx
2983
2984optional arguments::
2985
2986  -h, --help            show this help message and exit
2987  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2988                        list of datafiles to add as histograms
2989  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2990                        instrument parameter file, must be one for every
2991                        histogram
2992  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2993                        list of phases to add. phases are automatically
2994                        associated with all histograms given.
2995
2996    """
2997    proj = G2Project(gpxname=args.filename)
2998
2999    hist_objs = []
3000    if args.histograms:
3001        for h,i in zip(args.histograms,args.iparams):
3002            print("Adding histogram from",h,"with instparm ",i)
3003            hist_objs.append(proj.add_powder_histogram(h, i))
3004
3005    if args.phases: 
3006        for p in args.phases:
3007            print("Adding phase from",p)
3008            proj.add_phase(p, histograms=hist_objs)
3009        print('Linking phase(s) to histogram(s):')
3010        for h in hist_objs:
3011            print ('   '+h.name)
3012
3013    proj.save()
3014
3015commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
3016def add(args):
3017    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
3018
3019  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
3020                               [-i IPARAMS [IPARAMS ...]]
3021                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
3022                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
3023                               filename
3024
3025
3026positional arguments::
3027
3028  filename              the project file to open. Should end in .gpx
3029
3030optional arguments::
3031
3032  -h, --help            show this help message and exit
3033  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
3034                        list of datafiles to add as histograms
3035  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
3036                        instrument parameter file, must be one for every
3037                        histogram
3038  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
3039                        format hint for histogram import. Applies to all
3040                        histograms
3041  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
3042                        list of phases to add. phases are automatically
3043                        associated with all histograms given.
3044  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
3045                        format hint for phase import. Applies to all phases.
3046                        Example: -pf CIF
3047  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
3048                        list of histgram indices to associate with added
3049                        phases. If not specified, phases are associated with
3050                        all previously loaded histograms. Example: -l 2 3 4
3051   
3052    """
3053    proj = G2Project(args.filename)
3054
3055    if args.histograms:
3056        for h,i in zip(args.histograms,args.iparams):
3057            print("Adding histogram from",h,"with instparm ",i)
3058            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
3059
3060    if args.phases: 
3061        if not args.histlist:
3062            histlist = proj.histograms()
3063        else:
3064            histlist = [proj.histogram(i) for i in args.histlist]
3065
3066        for p in args.phases:
3067            print("Adding phase from",p)
3068            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
3069           
3070        if not args.histlist:
3071            print('Linking phase(s) to all histogram(s)')
3072        else:
3073            print('Linking phase(s) to histogram(s):')
3074            for h in histlist:
3075                print ('   '+h.name)
3076
3077    proj.save()
3078
3079
3080commandhelp["dump"] = "Shows the contents of a GSAS-II project"
3081def dump(args):
3082    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
3083
3084       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
3085
3086positional arguments::
3087
3088  files
3089
3090optional arguments::
3091
3092  -h, --help        show this help message and exit
3093  -d, --histograms  list histograms in files, overrides --raw
3094  -p, --phases      list phases in files, overrides --raw
3095  -r, --raw         dump raw file contents, default
3096 
3097    """
3098    if not args.histograms and not args.phases:
3099        args.raw = True
3100    if args.raw:
3101        import IPython.lib.pretty as pretty
3102
3103    for fname in args.files:
3104        if args.raw:
3105            proj, nameList = LoadDictFromProjFile(fname)
3106            print("file:", fname)
3107            print("names:", nameList)
3108            for key, val in proj.items():
3109                print(key, ":")
3110                pretty.pprint(val)
3111        else:
3112            proj = G2Project(fname)
3113            if args.histograms:
3114                hists = proj.histograms()
3115                for h in hists:
3116                    print(fname, "hist", h.id, h.name)
3117            if args.phases:
3118                phase_list = proj.phases()
3119                for p in phase_list:
3120                    print(fname, "phase", p.id, p.name)
3121
3122
3123commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
3124def IPyBrowse(args):
3125    """Load a .gpx file and then open a IPython shell to browse it::
3126
3127  usage: GSASIIscriptable.py browse [-h] files [files ...]
3128
3129positional arguments::
3130
3131  files       list of files to browse
3132
3133optional arguments::
3134
3135  -h, --help  show this help message and exit
3136
3137    """
3138    for fname in args.files:
3139        proj, nameList = LoadDictFromProjFile(fname)
3140        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
3141        GSASIIpath.IPyBreak_base(msg)
3142        break
3143
3144
3145commandhelp["refine"] = '''
3146Conducts refinements on GSAS-II projects according to a list of refinement
3147steps in a JSON dict
3148'''
3149def refine(args):
3150    """Implements the refine command-line subcommand:
3151    conducts refinements on GSAS-II projects according to a JSON refinement dict::
3152
3153        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
3154
3155positional arguments::
3156
3157  gpxfile      the project file to refine
3158  refinements  json file of refinements to apply. if not present refines file
3159               as-is
3160
3161optional arguments::
3162
3163  -h, --help   show this help message and exit
3164 
3165    """
3166    proj = G2Project(args.gpxfile)
3167    if args.refinements is None:
3168        proj.refine()
3169    else:
3170        import json
3171        with open(args.refinements) as refs:
3172            refs = json.load(refs)
3173        if type(refs) is not dict:
3174            raise G2ScriptException("Error: JSON object must be a dict.")
3175        if "code" in refs:
3176            print("executing code:\n|  ",'\n|  '.join(refs['code']))
3177            exec('\n'.join(refs['code']))
3178        proj.do_refinements(refs['refinements'])
3179
3180
3181commandhelp["seqrefine"] = "Not implemented. Placeholder for eventual sequential refinement implementation"
3182def seqrefine(args):
3183    """Future implementation for the seqrefine command-line subcommand """
3184    raise NotImplementedError("seqrefine is not yet implemented")
3185
3186
3187commandhelp["export"] = "Export phase as CIF"
3188def export(args):
3189    """Implements the export command-line subcommand: Exports phase as CIF::
3190
3191      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
3192
3193positional arguments::
3194
3195  gpxfile     the project file from which to export
3196  phase       identifier of phase to export
3197  exportfile  the .cif file to export to
3198
3199optional arguments::
3200
3201  -h, --help  show this help message and exit
3202
3203    """
3204    proj = G2Project(args.gpxfile)
3205    phase = proj.phase(args.phase)
3206    phase.export_CIF(args.exportfile)
3207
3208
3209def _args_kwargs(*args, **kwargs):
3210    return args, kwargs
3211
3212# A dictionary of the name of each subcommand, and a tuple
3213# of its associated function and a list of its arguments
3214# The arguments are passed directly to the add_argument() method
3215# of an argparse.ArgumentParser
3216
3217subcommands = {"create":
3218               (create, [_args_kwargs('filename',
3219                                      help='the project file to create. should end in .gpx'),
3220
3221                         _args_kwargs('-d', '--histograms',
3222                                      nargs='+',
3223                                      help='list of datafiles to add as histograms'),
3224                                     
3225                         _args_kwargs('-i', '--iparams',
3226                                      nargs='+',
3227                                      help='instrument parameter file, must be one'
3228                                           ' for every histogram'
3229                                      ),
3230
3231                         _args_kwargs('-p', '--phases',
3232                                      nargs='+',
3233                                      help='list of phases to add. phases are '
3234                                           'automatically associated with all '
3235                                           'histograms given.')]),
3236               "add": (add, [_args_kwargs('filename',
3237                                      help='the project file to open. Should end in .gpx'),
3238
3239                         _args_kwargs('-d', '--histograms',
3240                                      nargs='+',
3241                                      help='list of datafiles to add as histograms'),
3242                                     
3243                         _args_kwargs('-i', '--iparams',
3244                                      nargs='+',
3245                                      help='instrument parameter file, must be one'
3246                                           ' for every histogram'
3247                                      ),
3248                                     
3249                         _args_kwargs('-hf', '--histogramformat',
3250                                      help='format hint for histogram import. Applies to all'
3251                                           ' histograms'
3252                                      ),
3253
3254                         _args_kwargs('-p', '--phases',
3255                                      nargs='+',
3256                                      help='list of phases to add. phases are '
3257                                           'automatically associated with all '
3258                                           'histograms given.'),
3259
3260                         _args_kwargs('-pf', '--phaseformat',
3261                                      help='format hint for phase import. Applies to all'
3262                                           ' phases. Example: -pf CIF'
3263                                      ),
3264                                     
3265                         _args_kwargs('-l', '--histlist',
3266                                      nargs='+',
3267                                      help='list of histgram indices to associate with added'
3268                                           ' phases. If not specified, phases are'
3269                                           ' associated with all previously loaded'
3270                                           ' histograms. Example: -l 2 3 4')]),
3271                                           
3272               "dump": (dump, [_args_kwargs('-d', '--histograms',
3273                                     action='store_true',
3274                                     help='list histograms in files, overrides --raw'),
3275
3276                               _args_kwargs('-p', '--phases',
3277                                            action='store_true',
3278                                            help='list phases in files, overrides --raw'),
3279
3280                               _args_kwargs('-r', '--raw',
3281                                      action='store_true', help='dump raw file contents, default'),
3282
3283                               _args_kwargs('files', nargs='+')]),
3284
3285               "refine":
3286               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
3287                         _args_kwargs('refinements',
3288                                      help='JSON file of refinements to apply. if not present'
3289                                           ' refines file as-is',
3290                                      default=None,
3291                                      nargs='?')]),
3292
3293               "seqrefine": (seqrefine, []),
3294               "export": (export, [_args_kwargs('gpxfile',
3295                                                help='the project file from which to export'),
3296                                   _args_kwargs('phase', help='identifier of phase to export'),
3297                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
3298               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
3299                                                   help='list of files to browse')])}
3300
3301
3302def main():
3303    '''The command-line interface for calling GSASIIscriptable as a shell command,
3304    where it is expected to be called as::
3305
3306       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
3307
3308    The following subcommands are defined:
3309
3310        * create, see :func:`create`
3311        * add, see :func:`add`
3312        * dump, see :func:`dump`
3313        * refine, see :func:`refine`
3314        * seqrefine, see :func:`seqrefine`
3315        * export, :func:`export`
3316        * browse, see :func:`IPyBrowse`
3317
3318    .. seealso::
3319        :func:`create`
3320        :func:`add`
3321        :func:`dump`
3322        :func:`refine`
3323        :func:`seqrefine`
3324        :func:`export`
3325        :func:`IPyBrowse`
3326    '''
3327    parser = argparse.ArgumentParser(description=
3328        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
3329        )
3330    subs = parser.add_subparsers()
3331
3332    # Create all of the specified subparsers
3333    for name, (func, args) in subcommands.items():
3334        new_parser = subs.add_parser(name,help=commandhelp.get(name),
3335                                     description='Command "'+name+'" '+commandhelp.get(name))
3336        for listargs, kwds in args:
3337            new_parser.add_argument(*listargs, **kwds)
3338        new_parser.set_defaults(func=func)
3339
3340    # Parse and trigger subcommand
3341    result = parser.parse_args()
3342    result.func(result)
3343
3344if __name__ == '__main__':
3345    #fname='/tmp/corundum-template.gpx'
3346    #prj = G2Project(fname)
3347    main()
Note: See TracBrowser for help on using the repository browser.