source: trunk/GSASIIscriptable.py @ 3506

Last change on this file since 3506 was 3506, checked in by svnjenkins, 5 years ago

fix scriptable problem due to change in Restraints; update Tutorials; misc. fixes for Sphinx docs

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 133.9 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2018-07-25 05:12:20 +0000 (Wed, 25 Jul 2018) $
5# $Author: svnjenkins $
6# $Revision: 3506 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3506 2018-07-25 05:12:20Z svnjenkins $
9########### SVN repository information ###################
10#
11# TODO: integrate 2d data based on a model
12#
13"""
14*GSASIIscriptable: Scripting Interface*
15=======================================
16
17Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
18This file specifies several wrapper classes around GSAS-II data representations.
19They all inherit from :class:`G2ObjectWrapper`. The chief class is :class:`G2Project`,
20which represents an entire GSAS-II project and provides several methods to access
21phases, powder histograms, and execute Rietveld refinements. These routines can be
22accessed in two ways:
23the :ref:`CommandlineInterface` provides access a number of features without writing
24Python scripts of from the :ref:`API`, which allows much more versatile access from
25within Python.
26
27
28=====================
29Overview
30=====================
31The most commonly used routines are:
32
33:class:`G2Project`
34   Always needed: used to create a new project (.gpx) file or read in an existing one.
35
36:meth:`G2Project.add_powder_histogram`
37   Used to read in powder diffraction data to a project file.
38
39:meth:`G2Project.add_simulated_powder_histogram`
40   Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
41
42:meth:`G2Project.save`
43   Writes the current project to disk.
44
45:meth:`G2Project.do_refinements`
46   This is passed a list of dictionaries, where each dict defines a refinement step.
47   Passing a list with a single empty dict initiates a refinement with the current
48   parameters and flags.
49   
50:meth:`G2Project.set_refinement`
51   This is passed a single dict which is used to set parameters and flags.
52   These actions can be performed also in :meth:`G2Project.do_refinements`.
53
54Refinement dicts
55   A refinement dict sets up a single refinement step
56   (as used in :meth:`G2Project.do_refinements` and described in
57   :ref:`Project_dicts`).
58   It specifies parameter & refinement flag changes, which are usually followed by a refinement and
59   optionally by calls to locally-defined Python functions. The keys used in these dicts are
60   defined in the :ref:`Refinement_recipe` table, below.
61
62There are several ways to set parameters project using different level objects, as is
63are described in sections below, but the simplest way to access parameters and flags
64in a project is to use the above routines.
65
66=====================
67Refinement parameters
68=====================
69The most complex part of scripting GSAS-II comes in setting up the input to control refinements, which is
70described here.
71
72.. _Project_dicts:
73
74-----------------------------
75Project-level Parameter Dict
76-----------------------------
77
78As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
79which can be accessed individually by the objects that encapsulate individual phases and histograms
80but it will be usually simplest to create a composite dictionary
81that is used at the project-level. A dict is created with keys
82"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
83(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
84determine parameter values and will determine which parameters will be refined.
85
86The specific keys and subkeys that can be used are defined in tables
87:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
88
89Note that optionally a list of histograms and/or phases can be supplied in the call to
90:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
91phases and histograms.
92
93As an example:
94
95.. code-block::  python
96
97    pardict = {'set': { 'Limits': [0.8, 12.0],
98                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
99                       'Background': {'type': 'chebyschev', 'refine': True}},
100              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
101    my_project.set_refinement(pardict)
102   
103.. _Refinement_recipe:
104   
105------------------------
106Refinement recipe
107------------------------
108Building on the :ref:`Project_dicts`,
109it is possible to specify a sequence of refinement actions as a list of
110these dicts and supplying this list
111as an argument to :meth:`G2Project.do_refinements`.
112
113As an example, this code performs the same actions as in the example in the section above:
114
115.. code-block::  python
116   
117    pardict = {'set': { 'Limits': [0.8, 12.0],
118                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
119                       'Background': {'type': 'chebyschev', 'refine': True}},
120              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
121    my_project.do_refinements([pardict])
122
123However, in addition to setting a number of parameters, this example will perform a refinement as well.
124If more than one dict is specified in the list, as is done in this example,
125two refinement steps will be performed:
126
127.. code-block::  python
128
129    my_project.do_refinements([pardict,pardict1])
130
131
132The keys defined in the following table
133may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that now keys ``histograms``
134and ``phases`` are used to limit actions to specific sets of parameters within the project.
135
136========== ============================================================================
137key         explanation
138========== ============================================================================
139set                    Specifies a dict with keys and subkeys as described in the
140                       :ref:`Refinement_parameters_fmt` section. Items listed here
141                       will be set to be refined.
142clear                  Specifies a dict, as above for set, except that parameters are
143                       cleared and thus will not be refined.
144once                   Specifies a dict as above for set, except that parameters are
145                       set for the next cycle of refinement and are cleared once the
146                       refinement step is completed.
147skip                   Normally, once parameters are processed with a set/clear/once
148                       action(s), a refinement is started. If skip is defined as True
149                       (or any other value) the refinement step is not performed.
150output                 If a file name is specified for output is will be used to save
151                       the current refinement.
152histograms             Should contain a list of histogram(s) to be used for the
153                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
154                       :ref:`HAP_parameters_table`. Note that this will be
155                       ignored for :ref:`Phase_parameters_table`. Histograms may be
156                       specified as a list of strings [('PWDR ...'),...], indices
157                       [0,1,2] or as list of objects [hist1, hist2].
158phases                 Should contain a list of phase(s) to be used for the
159                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
160                       :ref:`HAP_parameters_table`. Note that this will be
161                       ignored for :ref:`Histogram_parameters_table`.
162                       Phases may be specified as a list of strings
163                       [('Phase name'),...], indices [0,1,2] or as list of objects
164                       [phase0, phase2].
165call                   Specifies a function to call after a refinement is completed.
166                       The value supplied can be the object (typically a function)
167                       that will be called or a string that will evaluate (in the
168                       namespace inside :meth:`G2Project.iter_refinements` where
169                       ``self`` references the project.)
170                       Nothing is called if this is not specified.
171callargs               Provides a list of arguments that will be passed to the function
172                       in call (if any). If call is defined and callargs is not, the
173                       current <tt>G2Project</tt> is passed as a single argument.
174========== ============================================================================
175
176An example that performs a series of refinement steps follows:
177
178.. code-block::  python
179
180    reflist = [
181            {"set": { "Limits": { "low": 0.7 },
182                      "Background": { "no. coeffs": 3,
183                                      "refine": True }}},
184            {"set": { "LeBail": True,
185                      "Cell": True }},
186            {"set": { "Sample Parameters": ["DisplaceX"]}},
187            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
188            {"set": { "Mustrain": { "type": "uniaxial",
189                                    "refine": "equatorial",
190                                    "direction": [0, 0, 1]}}},
191            {"set": { "Mustrain": { "type": "uniaxial",
192                                    "refine": "axial"}}},
193            {"clear": { "LeBail": True},
194             "set": { "Atoms": { "Mn": "X" }}},
195            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
196    my_project.do_refinements(reflist)
197   
198
199In this example, a separate refinement step will be performed for each dict in the list (since
200"skip" is not included).
201Note that in the second from last refinement step, parameters are both set and cleared.
202   
203.. _Refinement_parameters_kinds:
204
205----------------------------
206Refinement parameter types
207----------------------------
208
209Note that parameters and refinement flags used in GSAS-II fall into three classes:
210
211    * **Histogram**: There will be a set of these for each dataset loaded into a
212      project file. The parameters available depend on the type of histogram
213      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
214      include the overall scale factor, background, instrument and sample parameters;
215      see the :ref:`Histogram_parameters_table` table for a list of the histogram
216      parameters where access has been provided.
217     
218    * **Phase**: There will be a set of these for each phase loaded into a
219      project file. While some parameters are found in all types of phases,
220      others are only found in certain types (modulated, magnetic, protein...).
221      Typical phase parameters include unit cell lengths and atomic positions; see the
222      :ref:`Phase_parameters_table` table for a list of the phase     
223      parameters where access has been provided.
224     
225    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
226      that is associated with each phase, so that if there are ``N`` phases and ``M``
227      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
228      histograms are not linked to all phases.) Typical HAP parameters include the
229      phase fractions, sample microstrain and crystallite size broadening terms,
230      hydrostatic strain perturbations of the unit cell and preferred orientation
231      values.
232      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
233      been provided.
234
235.. _Refinement_parameters_fmt:
236
237=================================
238Specifying Refinement Parameters
239=================================
240
241Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
242where the details of these dicts are organized depends on the
243type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
244of keys (as described below) for each of the three types of parameters.
245
246.. _Histogram_parameters_table:
247
248--------------------
249Histogram parameters
250--------------------
251
252This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
253and :func:`G2PwdrData.clear_refinements`. As an example,
254
255.. code-block::  python
256
257   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
258                         "Sample Parameters": ["Scale"],
259                         "Limits": [10000, 40000]})
260
261With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
262``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
263key is used to define specific histograms. As an example:
264
265.. code-block::  python
266
267  gsas_proj.do_refinements([
268      {'set': {
269          'Background': {'no.coeffs': 3, 'refine': True},
270          'Sample Parameters': ['Scale'],
271          'Limits': [10000, 40000]},
272      'histograms': [1,2]}
273                            ])
274
275Note that below in the Instrument Parameters section,
276related profile parameters (such as U and V) are grouped together but
277separated by commas to save space in the table.
278
279.. tabularcolumns:: |l|l|p{3.5in}|
280
281===================== ====================  =================================================
282key                   subkey                explanation
283===================== ====================  =================================================
284Limits                                      The range of 2-theta (degrees) or TOF (in
285                                            microsec) range of values to use. Can
286                                            be either a dictionary of 'low' and/or 'high',
287                                            or a list of 2 items [low, high]
288\                     low                   Sets the low limit
289\                     high                  Sets the high limit
290
291Sample Parameters                           Should be provided as a **list** of subkeys
292                                            to set or clear, e.g. ['DisplaceX', 'Scale']
293\                     Absorption
294\                     Contrast
295\                     DisplaceX             Sample displacement along the X direction
296\                     DisplaceY             Sample displacement along the Y direction
297\                     Scale                 Histogram Scale factor
298
299Background                                  Sample background. If value is a boolean,
300                                            the background's 'refine' parameter is set
301                                            to the given boolean. Usually should be a
302                                            dictionary with any of the following keys:
303\                     type                  The background model, e.g. 'chebyschev'
304\                     refine                The value of the refine flag, boolean
305\                     no. coeffs            Number of coefficients to use, integer
306\                     coeffs                List of floats, literal values for background
307\                     FixedPoints           List of (2-theta, intensity) values for fixed points
308\                     fit fixed points      If True, triggers a fit to the fixed points to
309                                            be calculated. It is calculated when this key is
310                                            detected, regardless of calls to refine.
311
312Instrument Parameters                       As in Sample Paramters, provide as a **list** of
313                                            subkeys to
314                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
315\                     U, V, W               Gaussian peak profile terms
316\                     X, Y, Z               Lorentzian peak profile terms
317\                     alpha, beta-0,        TOF profile terms
318                      beta-1, beta-q,
319\                     sig-0, sig-1,         TOF profile terms
320                      sig-2, sig-q
321\                     difA, difB, difC      TOF Calibration constants
322\                     Zero                  Zero shift
323\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
324\                     Polariz.              Polarization parameter
325\                     Lam                   Lambda, the incident wavelength
326===================== ====================  =================================================
327
328.. _Phase_parameters_table:
329
330----------------
331Phase parameters
332----------------
333
334This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
335and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
336these parameters should be placed inside a dict with a key
337``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
338key is used to define specific phase(s).
339
340
341.. tabularcolumns:: |l|p{4.5in}|
342
343======= ==========================================================
344key                   explanation
345======= ==========================================================
346Cell                  Whether or not to refine the unit cell.
347Atoms                 Dictionary of atoms and refinement flags.
348                      Each key should be an atom label, e.g.
349                      'O3', 'Mn5', and each value should be
350                      a string defining what values to refine.
351                      Values can be any combination of 'F'
352                      for fractional occupancy, 'X' for position,
353                      and 'U' for Debye-Waller factor
354LeBail                Enables LeBail intensity extraction.
355======= ==========================================================
356
357
358.. _HAP_parameters_table:
359
360
361Histogram-and-phase parameters
362------------------------------
363
364This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
365and :func:`G2Phase.clear_HAP_refinements`. When supplied to
366:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
367``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
368unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
369
370.. tabularcolumns:: |l|l|p{3.5in}|
371
372=============  ==========  ============================================================
373key             subkey                 explanation
374=============  ==========  ============================================================
375Babinet                                Should be a **list** of the following
376                                       subkeys. If not, assumes both
377                                       BabA and BabU
378\               BabA
379\               BabU
380Extinction                             Boolean, True to refine.
381HStrain                                Boolean, True to refine all appropriate
382                                       $D_ij$ terms.
383Mustrain
384\               type                   Mustrain model. One of 'isotropic',
385                                       'uniaxial', or 'generalized'. Should always
386                                       be specified.
387\              direction               For uniaxial only. A list of three
388                                       integers,
389                                       the [hkl] direction of the axis.
390\               refine                 Usually boolean, set to True to refine.
391                                       or False to clear.
392                                       For uniaxial model, can specify a value
393                                       of 'axial' or 'equatorial' to set that flag
394                                       to True or a single
395                                       boolean sets both axial and equatorial.
396Size                                   
397\               type                   Size broadening model. One of 'isotropic',
398                                       'uniaxial', or 'ellipsoid'. Should always
399                                       be specified.
400\              direction               For uniaxial only. A list of three
401                                       integers,
402                                       the [hkl] direction of the axis.
403\               refine                 Boolean, True to refine.
404\               value                  float, size value in microns
405Pref.Ori.                              Boolean, True to refine
406Show                                   Boolean, True to refine
407Use                                    Boolean, True to refine
408Scale                                  Phase fraction; Boolean, True to refine
409=============  ==========  ============================================================
410
411------------------------
412Histogram/Phase objects
413------------------------
414Each phase and powder histogram in a :class:`G2Project` object has an associated
415object. Parameters within each individual object can be turned on and off by calling
416:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
417for histogram parameters;
418:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
419for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
420:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
421
422.. code-block::  python
423
424    params = { 'Limits': [0.8, 12.0],
425               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
426               'Background': {'type': 'chebyschev', 'refine': True}}
427    some_histogram.set_refinements(params)
428
429Likewise to turn refinement flags on, use code such as this:
430
431.. code-block::  python
432
433    params = { 'Instrument Parameters': ['U', 'V', 'W']}
434    some_histogram.set_refinements(params)
435
436and to turn these refinement flags, off use this (Note that the
437``.clear_refinements()`` methods will usually will turn off refinement even
438if a refinement parameter is set in the dict to True.):
439
440.. code-block::  python
441
442    params = { 'Instrument Parameters': ['U', 'V', 'W']}
443    some_histogram.clear_refinements(params)
444
445For phase parameters, use code such as this:
446   
447.. code-block::  python
448
449    params = { 'LeBail': True, 'Cell': True,
450               'Atoms': { 'Mn1': 'X',
451                          'O3': 'XU',
452                          'V4': 'FXU'}}
453    some_histogram.set_refinements(params)
454
455
456and here is an example for HAP parameters:
457
458.. code-block::  python
459
460    params = { 'Babinet': 'BabA',
461               'Extinction': True,
462               'Mustrain': { 'type': 'uniaxial',
463                             'direction': [0, 0, 1],
464                             'refine': True}}
465    some_phase.set_HAP_refinements(params)
466
467Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
468
469.. _CommandlineInterface:
470
471=======================================
472GSASIIscriptable Command-line Interface
473=======================================
474
475The routines described above are intended to be called from a Python script, but an
476alternate way to access some of the same functionality is to
477invoke the ``GSASIIscriptable.py`` script from
478the command line usually from within a shell script or batch file. This
479will usually be done with a command such as::
480
481       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
482
483    The following subcommands are defined:
484
485        * create, see :func:`create`
486        * add, see :func:`add`
487        * dump, see :func:`dump`
488        * refine, see :func:`refine`
489        * seqrefine, see :func:`seqrefine`
490        * export, :func:`export`
491        * browse, see :func:`IPyBrowse`
492       
493Run::
494
495   python GSASIIscriptable.py --help
496
497to show the available subcommands, and inspect each subcommand with
498`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
499
500.. _JsonFormat:
501
502-------------------------
503Parameters in JSON files
504-------------------------
505
506The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
507a JSON format file
508(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
509This dict may have two keys:
510
511refinements:
512  This defines the a set of refinement steps in a JSON representation of a
513  :ref:`Refinement_recipe` list.
514
515code:
516  This optionally defines Python code that will be executed after the project is loaded,
517  but before the refinement is started. This can be used to execute Python code to change
518  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
519  project object is accessed with variable ``proj``) or to define code that will be called
520  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
521   
522JSON website: `Introducing JSON <http://json.org/>`_.
523
524.. _API:
525
526============================================================
527GSASIIscriptable Application Layer (API)
528============================================================
529
530The large number of classes and modules in this module are described below.
531Most commonly a script will create a G2Project object using :class:`G2Project` and then
532perform actions such as
533adding a histogram (method :meth:`G2Project.add_powder_histogram`),
534adding a phase (method :meth:`G2Project.add_phase`),
535or setting parameters and performing a refinement
536(method :meth:`G2Project.do_refinements`).
537
538In some cases, it may be easier to use
539methods inside :class:`G2PwdrData` or :class:`G2Phase` or these objects
540may offer more options.
541
542---------------------------------------------------------------
543Complete Documentation: All classes and functions
544---------------------------------------------------------------
545"""
546from __future__ import division, print_function
547import argparse
548import os.path as ospath
549import datetime as dt
550import sys
551import platform
552if '2' in platform.python_version_tuple()[0]:
553    import cPickle
554    strtypes = (str,unicode)
555else:
556    import _pickle as cPickle
557    strtypes = (str,bytes)
558import imp
559import copy
560import os
561import random as ran
562
563import numpy.ma as ma
564import scipy.interpolate as si
565import numpy as np
566import scipy as sp
567
568import GSASIIpath
569GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
570import GSASIIobj as G2obj
571import GSASIIpwd as G2pwd
572import GSASIIstrMain as G2strMain
573import GSASIIspc as G2spc
574import GSASIIElem as G2elem
575
576
577# Delay imports to not slow down small scripts
578G2fil = None
579PwdrDataReaders = []
580PhaseReaders = []
581exportersByExtension = {}
582'''Specifies the list of extensions that are supported for Powder data export'''
583
584def LoadG2fil():
585    """Delay importing this module, it is slow"""
586    global G2fil
587    if G2fil is None:
588        import GSASIIfiles
589        G2fil = GSASIIfiles
590        global PwdrDataReaders
591        global PhaseReaders
592        PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
593        PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
594        AllExporters = G2fil.LoadExportRoutines(None)
595        global exportersByExtension
596        exportersByExtension = {}
597        for obj in AllExporters:
598            try:
599                obj.Writer
600            except AttributeError:
601                continue
602            for typ in obj.exporttype:
603                if typ not in exportersByExtension:
604                    exportersByExtension[typ] = {obj.extension:obj}
605                else:
606                    exportersByExtension[typ][obj.extension] = obj
607
608def LoadDictFromProjFile(ProjFile):
609    '''Read a GSAS-II project file and load items to dictionary
610   
611    :param str ProjFile: GSAS-II project (name.gpx) full file name
612    :returns: Project,nameList, where
613
614      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
615        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
616        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
617      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
618
619    Example for fap.gpx::
620
621      Project = {                 #NB:dict order is not tree order
622        u'Phases':{'data':None,'fap':{phase dict}},
623        u'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
624        u'Rigid bodies':{'data': {rigid body dict}},
625        u'Covariance':{'data':{covariance data dict}},
626        u'Controls':{'data':{controls data dict}},
627        u'Notebook':{'data':[notebook list]},
628        u'Restraints':{'data':{restraint data dict}},
629        u'Constraints':{'data':{constraint data dict}}]
630        }
631      nameList = [                #NB: reproduces tree order
632        [u'Notebook',],
633        [u'Controls',],
634        [u'Covariance',],
635        [u'Constraints',],
636        [u'Restraints',],
637        [u'Rigid bodies',],
638        [u'PWDR FAP.XRA Bank 1',
639             u'Comments',
640             u'Limits',
641             u'Background',
642             u'Instrument Parameters',
643             u'Sample Parameters',
644             u'Peak List',
645             u'Index Peak List',
646             u'Unit Cells List',
647             u'Reflection Lists'],
648        [u'Phases', u'fap']
649        ]
650    '''
651    # Let IOError be thrown if file does not exist
652    # if not ospath.exists(ProjFile):
653    #     print ('\n*** Error attempt to open project file that does not exist:\n   '+
654    #         str(ProjFile))
655    #     return
656    file = open(ProjFile,'rb')
657    # print('loading from file: {}'.format(ProjFile))
658    Project = {}
659    nameList = []
660    try:
661        while True:
662            try:
663                data = cPickle.load(file)
664            except EOFError:
665                break
666            datum = data[0]
667            Project[datum[0]] = {'data':datum[1]}
668            nameList.append([datum[0],])
669            for datus in data[1:]:
670                Project[datum[0]][datus[0]] = datus[1]
671                nameList[-1].append(datus[0])
672        file.close()
673        # print('project load successful')
674    except:
675        raise IOError("Error reading file "+str(ProjFile)+". This is not a GSAS-II .gpx file")
676    finally:
677        file.close()
678    return Project,nameList
679
680def SaveDictToProjFile(Project,nameList,ProjFile):
681    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
682
683    :param dict Project: representation of gpx file following the GSAS-II
684        tree structure as described for LoadDictFromProjFile
685    :param list nameList: names of main tree entries & subentries used to reconstruct project file
686    :param str ProjFile: full file name for output project.gpx file (including extension)
687    '''
688    file = open(ProjFile,'wb')
689    try:
690        for name in nameList:
691            data = []
692            item = Project[name[0]]
693            data.append([name[0],item['data']])
694            for item2 in name[1:]:
695                data.append([item2,item[item2]])
696            cPickle.dump(data,file,1)
697    finally:
698        file.close()
699
700# def ImportPowder(reader,filename):
701#     '''Use a reader to import a powder diffraction data file
702
703#     :param str reader: a scriptable reader
704#     :param str filename: full name of powder data file; can be "multi-Bank" data
705
706#     :returns: list rdlist: list of reader objects containing powder data, one for each
707#         "Bank" of data encountered in file. Items in reader object of interest are:
708
709#           * rd.comments: list of str: comments found on powder file
710#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
711#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
712#     '''
713#     rdfile,rdpath,descr = imp.find_module(reader)
714#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
715#     rd = rdclass.GSAS_ReaderClass()
716#     if not rd.scriptable:
717#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
718#         return None
719#     rdlist = []
720#     if rd.ContentsValidator(filename):
721#         repeat = True
722#         rdbuffer = {} # create temporary storage for file reader
723#         block = 0
724#         while repeat: # loop if the reader asks for another pass on the file
725#             block += 1
726#             repeat = False
727#             rd.objname = ospath.basename(filename)
728#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
729#             if flag:
730#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
731#                 if rd.repeat:
732#                     repeat = True
733#         return rdlist
734#     print(rd.errors)
735#     return None
736
737def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
738    '''Create an initial Histogram dictionary
739
740    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
741    '''
742    if dType in ['SXC','SNC']:
743        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
744            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
745            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
746            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
747            'Flack':[0.0,False]}
748    elif dType == 'SNT':
749        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
750            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
751            'Extinction':['Lorentzian','None', {
752            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
753    elif 'P' in dType:
754        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
755            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
756            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
757                [1.,1.,1.,0.,0.,0.],6*[False,]],
758            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
759                NShkl*[0.01,],NShkl*[False,]],
760            'HStrain':[NDij*[0.0,],NDij*[False,]],
761            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
762
763
764def PreSetup(data):
765    '''Create part of an initial (empty) phase dictionary
766
767    from GSASIIphsGUI.py, near end of UpdatePhaseData
768
769    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
770    '''
771    if 'RBModels' not in data:
772        data['RBModels'] = {}
773    if 'MCSA' not in data:
774        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
775    if 'dict' in str(type(data['MCSA']['Results'])):
776        data['MCSA']['Results'] = []
777    if 'Modulated' not in data['General']:
778        data['General']['Modulated'] = False
779#    if 'modulated' in data['General']['Type']:
780#        data['General']['Modulated'] = True
781#        data['General']['Type'] = 'nuclear'
782
783
784def SetupGeneral(data, dirname):
785    """Helps initialize phase data.
786
787    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
788
789    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
790    """
791    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
792                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
793    generalData = data['General']
794    atomData = data['Atoms']
795    generalData['AtomTypes'] = []
796    generalData['Isotopes'] = {}
797
798    if 'Isotope' not in generalData:
799        generalData['Isotope'] = {}
800    if 'Data plot type' not in generalData:
801        generalData['Data plot type'] = 'Mustrain'
802    if 'POhkl' not in generalData:
803        generalData['POhkl'] = [0,0,1]
804    if 'Map' not in generalData:
805        generalData['Map'] = mapDefault.copy()
806    if 'Flip' not in generalData:
807        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
808            'k-factor':0.1,'k-Max':20.,}
809    if 'testHKL' not in generalData['Flip']:
810        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
811    if 'doPawley' not in generalData:
812        generalData['doPawley'] = False     #ToDo: change to ''
813    if 'Pawley dmin' not in generalData:
814        generalData['Pawley dmin'] = 1.0
815    if 'Pawley neg wt' not in generalData:
816        generalData['Pawley neg wt'] = 0.0
817    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
818        'MCSA controls' not in generalData:
819        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
820        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
821        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
822    if 'AtomPtrs' not in generalData:
823        generalData['AtomPtrs'] = [3,1,7,9]
824        if generalData['Type'] == 'macromolecular':
825            generalData['AtomPtrs'] = [6,4,10,12]
826        elif generalData['Type'] == 'magnetic':
827            generalData['AtomPtrs'] = [3,1,10,12]
828    if generalData['Modulated']:
829        generalData['Type'] = 'nuclear'
830        if 'Super' not in generalData:
831            generalData['Super'] = 1
832            generalData['SuperVec'] = [[0,0,.1],False,4]
833            generalData['SSGData'] = {}
834        if '4DmapData' not in generalData:
835            generalData['4DmapData'] = mapDefault.copy()
836            generalData['4DmapData'].update({'MapType':'Fobs'})
837    if 'Modulated' not in generalData:
838        generalData['Modulated'] = False
839    if 'HydIds' not in generalData:
840        generalData['HydIds'] = {}
841    cx,ct,cs,cia = generalData['AtomPtrs']
842    generalData['NoAtoms'] = {}
843    generalData['BondRadii'] = []
844    generalData['AngleRadii'] = []
845    generalData['vdWRadii'] = []
846    generalData['AtomMass'] = []
847    generalData['Color'] = []
848    if generalData['Type'] == 'magnetic':
849        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
850        landeg = generalData.get('Lande g',[])
851    generalData['Mydir'] = dirname
852    badList = {}
853    for iat,atom in enumerate(atomData):
854        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
855        if generalData['AtomTypes'].count(atom[ct]):
856            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
857        elif atom[ct] != 'UNK':
858            Info = G2elem.GetAtomInfo(atom[ct])
859            if not Info:
860                if atom[ct] not in badList:
861                    badList[atom[ct]] = 0
862                badList[atom[ct]] += 1
863                atom[ct] = 'UNK'
864                continue
865            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
866            generalData['AtomTypes'].append(atom[ct])
867            generalData['Z'] = Info['Z']
868            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
869            generalData['BondRadii'].append(Info['Drad'])
870            generalData['AngleRadii'].append(Info['Arad'])
871            generalData['vdWRadii'].append(Info['Vdrad'])
872            if atom[ct] in generalData['Isotope']:
873                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
874                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
875                    generalData['Isotope'][atom[ct]] = isotope
876                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
877            else:
878                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
879                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
880                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
881                    generalData['Isotope'][atom[ct]] = isotope
882                generalData['AtomMass'].append(Info['Mass'])
883            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
884            generalData['Color'].append(Info['Color'])
885            if generalData['Type'] == 'magnetic':
886                if len(landeg) < len(generalData['AtomTypes']):
887                    landeg.append(2.0)
888    if generalData['Type'] == 'magnetic':
889        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
890
891    if badList:
892        msg = 'Warning: element symbol(s) not found:'
893        for key in badList:
894            msg += '\n\t' + key
895            if badList[key] > 1:
896                msg += ' (' + str(badList[key]) + ' times)'
897        raise G2ScriptException("Phase error:\n" + msg)
898        # wx.MessageBox(msg,caption='Element symbol error')
899    F000X = 0.
900    F000N = 0.
901    for i,elem in enumerate(generalData['AtomTypes']):
902        F000X += generalData['NoAtoms'][elem]*generalData['Z']
903        isotope = generalData['Isotope'][elem]
904        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
905    generalData['F000X'] = F000X
906    generalData['F000N'] = F000N
907    import GSASIImath as G2mth
908    generalData['Mass'] = G2mth.getMass(generalData)
909
910
911def make_empty_project(author=None, filename=None):
912    """Creates an dictionary in the style of GSASIIscriptable, for an empty
913    project.
914
915    If no author name or filename is supplied, 'no name' and
916    <current dir>/test_output.gpx are used , respectively.
917
918    Returns: project dictionary, name list
919
920    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
921    """
922    if not filename:
923        filename = 'test_output.gpx'
924    filename = os.path.abspath(filename)
925    gsasii_version = str(GSASIIpath.GetVersionNumber())
926    LoadG2fil()
927    import matplotlib as mpl
928    python_library_versions = G2fil.get_python_versions([mpl, np, sp])
929
930    controls_data = dict(G2obj.DefaultControls)
931    controls_data['LastSavedAs'] = filename
932    controls_data['LastSavedUsing'] = gsasii_version
933    controls_data['PythonVersions'] = python_library_versions
934    if author:
935        controls_data['Author'] = author
936
937    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
938                                       'Global': []}},
939              'Controls': {'data': controls_data},
940              u'Covariance': {'data': {}},
941              u'Notebook': {'data': ['']},
942              u'Restraints': {'data': {}},
943              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
944                                'Residue': {'AtInfo': {}},
945                                'Vector':  {'AtInfo': {}}}}}
946
947    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
948             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
949
950    return output, names
951
952
953class G2ImportException(Exception):
954    pass
955
956class G2ScriptException(Exception):
957    pass
958
959def import_generic(filename, readerlist, fmthint=None):
960    """Attempt to import a filename, using a list of reader objects.
961
962    Returns the first reader object which worked."""
963    # Translated from OnImportGeneric method in GSASII.py
964    primaryReaders, secondaryReaders = [], []
965    for reader in readerlist:
966        if fmthint is not None and fmthint not in reader.formatName: continue
967        flag = reader.ExtensionValidator(filename)
968        if flag is None:
969            secondaryReaders.append(reader)
970        elif flag:
971            primaryReaders.append(reader)
972    if not secondaryReaders and not primaryReaders:
973        raise G2ImportException("Could not read file: ", filename)
974
975    with open(filename, 'Ur') as fp:
976        rd_list = []
977
978        for rd in primaryReaders + secondaryReaders:
979            # Initialize reader
980            rd.selections = []
981            rd.dnames = []
982            rd.ReInitialize()
983            # Rewind file
984            rd.errors = ""
985            if not rd.ContentsValidator(filename):
986                # Report error
987                pass
988            if len(rd.selections) > 1:
989                # Select data?
990                # GSASII.py:543
991                raise G2ImportException("Not sure what data to select")
992
993            block = 0
994            rdbuffer = {}
995            repeat = True
996            while repeat:
997                repeat = False
998                block += 1
999                rd.objname = os.path.basename(filename)
1000                try:
1001                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
1002                except:
1003                    flag = False
1004                if flag:
1005                    # Omitting image loading special cases
1006                    rd.readfilename = filename
1007                    rd_list.append(copy.deepcopy(rd))
1008                    repeat = rd.repeat
1009                else:
1010                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
1011            if rd_list:
1012                if rd.warnings:
1013                    print("Read warning by", rd.formatName, "reader:",
1014                          rd.warnings, file=sys.stderr)
1015                else:
1016                    print("{} read by Reader {}\n".format(filename,rd.formatName))                   
1017                return rd_list
1018    raise G2ImportException("No reader could read file: " + filename)
1019
1020
1021def load_iprms(instfile, reader):
1022    """Loads instrument parameters from a file, and edits the
1023    given reader.
1024
1025    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1026    """
1027    LoadG2fil()
1028    ext = os.path.splitext(instfile)[1]
1029
1030    if ext.lower() == '.instprm':
1031        # New GSAS File, load appropriately
1032        with open(instfile) as f:
1033            lines = f.readlines()
1034        bank = reader.powderentry[2]
1035        numbanks = reader.numbanks
1036        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1037
1038        reader.instfile = instfile
1039        reader.instmsg = 'GSAS-II file' + instfile
1040        return iparms
1041    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1042        raise ValueError('Expected .prm file, found: ', instfile)
1043
1044    # It's an old GSAS file, load appropriately
1045    Iparm = {}
1046    with open(instfile, 'Ur') as fp:
1047        for line in fp:
1048            if '#' in line:
1049                continue
1050            Iparm[line[:12]] = line[12:-1]
1051    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1052    if ibanks == 1:
1053        reader.instbank = 1
1054        reader.powderentry[2] = 1
1055        reader.instfile = instfile
1056        reader.instmsg = instfile + ' bank ' + str(reader.instbank)
1057        return G2fil.SetPowderInstParms(Iparm, reader)
1058    # TODO handle >1 banks
1059    raise NotImplementedError("Check GSASIIfiles.py: ReadPowderInstprm")
1060
1061def load_pwd_from_reader(reader, instprm, existingnames=[]):
1062    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1063
1064    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1065
1066    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1067    """
1068    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1069    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1070
1071    try:
1072        Iparm1, Iparm2 = instprm
1073    except ValueError:
1074        Iparm1, Iparm2 = load_iprms(instprm, reader)
1075
1076    Ymin = np.min(reader.powderdata[1])
1077    Ymax = np.max(reader.powderdata[1])
1078    valuesdict = {'wtFactor': 1.0,
1079                  'Dummy': False,
1080                  'ranId': ran.randint(0, sys.maxsize),
1081                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1082                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1083                  'Yminmax': [Ymin, Ymax]}
1084    reader.Sample['ranId'] = valuesdict['ranId']
1085
1086    # Ending keys:
1087    # [u'Reflection Lists',
1088    #  u'Limits',
1089    #  'data',
1090    #  u'Index Peak List',
1091    #  u'Comments',
1092    #  u'Unit Cells List',
1093    #  u'Sample Parameters',
1094    #  u'Peak List',
1095    #  u'Background',
1096    #  u'Instrument Parameters']
1097    Tmin = np.min(reader.powderdata[0])
1098    Tmax = np.max(reader.powderdata[0])
1099
1100    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1101                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1102
1103    output_dict = {u'Reflection Lists': {},
1104                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1105                   u'data': [valuesdict, reader.powderdata, HistName],
1106                   u'Index Peak List': [[], []],
1107                   u'Comments': reader.comments,
1108                   u'Unit Cells List': [],
1109                   u'Sample Parameters': reader.Sample,
1110                   u'Peak List': {'peaks': [], 'sigDict': {}},
1111                   u'Background': reader.pwdparms.get('Background', default_background),
1112                   u'Instrument Parameters': [Iparm1, Iparm2],
1113                   }
1114
1115    names = [u'Comments',
1116             u'Limits',
1117             u'Background',
1118             u'Instrument Parameters',
1119             u'Sample Parameters',
1120             u'Peak List',
1121             u'Index Peak List',
1122             u'Unit Cells List',
1123             u'Reflection Lists']
1124
1125    # TODO controls?? GSASII.py:1664-7
1126
1127    return HistName, [HistName] + names, output_dict
1128
1129
1130def _deep_copy_into(from_, into):
1131    """Helper function for reloading .gpx file. See G2Project.reload()
1132
1133    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1134    """
1135    if isinstance(from_, dict) and isinstance(into, dict):
1136        combined_keys = set(from_.keys()).union(into.keys())
1137        for key in combined_keys:
1138            if key in from_ and key in into:
1139                both_dicts = (isinstance(from_[key], dict)
1140                              and isinstance(into[key], dict))
1141                both_lists = (isinstance(from_[key], list)
1142                              and isinstance(into[key], list))
1143                if both_dicts or both_lists:
1144                    _deep_copy_into(from_[key], into[key])
1145                else:
1146                    into[key] = from_[key]
1147            elif key in from_:
1148                into[key] = from_[key]
1149            else:  # key in into
1150                del into[key]
1151    elif isinstance(from_, list) and isinstance(into, list):
1152        if len(from_) == len(into):
1153            for i in range(len(from_)):
1154                both_dicts = (isinstance(from_[i], dict)
1155                              and isinstance(into[i], dict))
1156                both_lists = (isinstance(from_[i], list)
1157                              and isinstance(into[i], list))
1158                if both_dicts or both_lists:
1159                    _deep_copy_into(from_[i], into[i])
1160                else:
1161                    into[i] = from_[i]
1162        else:
1163            into[:] = from_
1164
1165
1166class G2ObjectWrapper(object):
1167    """Base class for all GSAS-II object wrappers.
1168
1169    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1170    of overrides are implemented so that the wrapper behaves like a dictionary.
1171
1172    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1173    """
1174    def __init__(self, datadict):
1175        self.data = datadict
1176
1177    def __getitem__(self, key):
1178        return self.data[key]
1179
1180    def __setitem__(self, key, value):
1181        self.data[key] = value
1182
1183    def __contains__(self, key):
1184        return key in self.data
1185
1186    def get(self, k, d=None):
1187        return self.data.get(k, d)
1188
1189    def keys(self):
1190        return self.data.keys()
1191
1192    def values(self):
1193        return self.data.values()
1194
1195    def items(self):
1196        return self.data.items()
1197
1198
1199class G2Project(G2ObjectWrapper):
1200    """
1201    Represents an entire GSAS-II project.
1202
1203    :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1204            creates an empty project.
1205    :param str author: Author's name (not yet implemented)
1206    :param str newgpx: The filename the project should be saved to in
1207            the future. If both newgpx and gpxfile are present, the project is
1208            loaded from the gpxfile, then when saved will be written to newgpx.
1209    :param str filename: Name to be used to save the project. Has same function as
1210            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1211            is preferred over filename.
1212
1213    There are two ways to initialize this object:
1214
1215    >>> # Load an existing project file
1216    >>> proj = G2Project('filename.gpx')
1217   
1218    >>> # Create a new project
1219    >>> proj = G2Project(newgpx='new_file.gpx')
1220   
1221    Histograms can be accessed easily.
1222
1223    >>> # By name
1224    >>> hist = proj.histogram('PWDR my-histogram-name')
1225   
1226    >>> # Or by index
1227    >>> hist = proj.histogram(0)
1228    >>> assert hist.id == 0
1229   
1230    >>> # Or by random id
1231    >>> assert hist == proj.histogram(hist.ranId)
1232
1233    Phases can be accessed the same way.
1234
1235    >>> phase = proj.phase('name of phase')
1236
1237    New data can also be loaded via :meth:`~G2Project.add_phase` and
1238    :meth:`~G2Project.add_powder_histogram`.
1239
1240    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1241                                         'instrument_parameters.prm')
1242    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1243
1244    Parameters for Rietveld refinement can be turned on and off as well.
1245    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1246    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1247    """
1248    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1249        if filename is not None and newgpx is not None:
1250            raise G2ScriptException('Do not use filename and newgpx together')
1251        elif newgpx is not None:
1252            filename = newgpx
1253        if gpxfile is None:
1254            filename = os.path.abspath(os.path.expanduser(filename))
1255            self.filename = filename
1256            self.data, self.names = make_empty_project(author=author, filename=filename)
1257        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1258                                       # filename is valid, etc.
1259            if isinstance(filename, str): 
1260                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1261            else: 
1262                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1263            # TODO set author
1264            self.data, self.names = LoadDictFromProjFile(gpxfile)
1265            self.update_ids()
1266        else:
1267            raise ValueError("Not sure what to do with gpxfile")
1268
1269    @classmethod
1270    def from_dict_and_names(cls, gpxdict, names, filename=None):
1271        """Creates a :class:`G2Project` directly from
1272        a dictionary and a list of names. If in doubt, do not use this.
1273
1274        :returns: a :class:`G2Project`
1275        """
1276        out = cls()
1277        if filename:
1278            filename = os.path.abspath(os.path.expanduser(filename))
1279            out.filename = filename
1280            gpxdict['Controls']['data']['LastSavedAs'] = filename
1281        else:
1282            try:
1283                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1284            except KeyError:
1285                out.filename = None
1286        out.data = gpxdict
1287        out.names = names
1288
1289    def save(self, filename=None):
1290        """Saves the project, either to the current filename, or to a new file.
1291
1292        Updates self.filename if a new filename provided"""
1293        # TODO update LastSavedUsing ?
1294        if filename:
1295            filename = os.path.abspath(os.path.expanduser(filename))
1296            self.data['Controls']['data']['LastSavedAs'] = filename
1297            self.filename = filename
1298        elif not self.filename:
1299            raise AttributeError("No file name to save to")
1300        SaveDictToProjFile(self.data, self.names, self.filename)
1301
1302    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None):
1303        """Loads a powder data histogram into the project.
1304
1305        Automatically checks for an instrument parameter file, or one can be
1306        provided. Note that in unix fashion, "~" can be used to indicate the
1307        home directory (e.g. ~/G2data/data.fxye).
1308
1309        :param str datafile: The powder data file to read, a filename.
1310        :param str iparams: The instrument parameters file, a filename.
1311        :param list phases: Phases to link to the new histogram
1312        :param str fmthint: If specified, only importers where the format name
1313          (reader.formatName, as shown in Import menu) contains the
1314          supplied string will be tried as importers. If not specified, all
1315          importers consistent with the file extension will be tried
1316          (equivalent to "guess format" in menu).
1317
1318        :returns: A :class:`G2PwdrData` object representing
1319            the histogram
1320        """
1321        LoadG2fil()
1322        datafile = os.path.abspath(os.path.expanduser(datafile))
1323        iparams = os.path.abspath(os.path.expanduser(iparams))
1324        pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint)
1325        histname, new_names, pwdrdata = load_pwd_from_reader(
1326                                          pwdrreaders[0], iparams,
1327                                          [h.name for h in self.histograms()])
1328        if histname in self.data:
1329            print("Warning - redefining histogram", histname)
1330        elif self.names[-1][0] == 'Phases':
1331            self.names.insert(-1, new_names)
1332        else:
1333            self.names.append(new_names)
1334        self.data[histname] = pwdrdata
1335        self.update_ids()
1336
1337        for phase in phases:
1338            phase = self.phase(phase)
1339            self.link_histogram_phase(histname, phase)
1340
1341        return self.histogram(histname)
1342
1343    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1344                                       wavelength=None, scale=None, phases=[]):
1345        """Loads a powder data histogram into the project.
1346
1347        Requires an instrument parameter file.
1348        Note that in unix fashion, "~" can be used to indicate the
1349        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1350        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1351        as the instrument type.
1352
1353        :param str histname: A name for the histogram to be created.
1354        :param str iparams: The instrument parameters file, a filename.
1355        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1356        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1357        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1358        :param float wavelength: Wavelength for CW instruments, overriding the value
1359           in the instrument parameters file if specified.
1360        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1361           simulated noise is added to the pattern, so that if the maximum intensity is
1362           small, the noise will mask the computed pattern. The scale
1363           needs to be a large number for CW neutrons.
1364           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1365        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1366           all defined phases.
1367
1368        :returns: A :class:`G2PwdrData` object representing the histogram
1369        """
1370        LoadG2fil()
1371        iparams = os.path.abspath(os.path.expanduser(iparams))
1372        if not os.path.exists(iparams):
1373            raise G2ScriptException("File does not exist:"+iparams)
1374        rd = G2obj.ImportPowderData( # Initialize a base class reader
1375            extensionlist=tuple(),
1376            strictExtension=False,
1377            formatName = 'Simulate dataset',
1378            longFormatName = 'Compute a simulated pattern')
1379        rd.powderentry[0] = '' # no filename
1380        rd.powderentry[2] = 1 # only one bank
1381        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1382        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1383        if Tmax < Tmin:
1384            Tmin,Tmax = Tmax,Tmin
1385        Tstep = abs(Tstep)
1386        if 'TOF' in rd.idstring:
1387                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1388                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1389                N = len(x)
1390        else:           
1391                N = int((Tmax-Tmin)/Tstep)+1
1392                x = np.linspace(Tmin,Tmax,N,True)
1393                N = len(x)
1394        if N < 3:
1395            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1396        rd.powderdata = [
1397            np.array(x), # x-axis values
1398            np.zeros_like(x), # powder pattern intensities
1399            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1400            np.zeros_like(x), # calc. intensities (zero)
1401            np.zeros_like(x), # calc. background (zero)
1402            np.zeros_like(x), # obs-calc profiles
1403            ]
1404        Tmin = rd.powderdata[0][0]
1405        Tmax = rd.powderdata[0][-1]
1406        rd.idstring = histname
1407        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1408                                                            [h.name for h in self.histograms()])
1409        if histname in self.data:
1410            print("Warning - redefining histogram", histname)
1411        elif self.names[-1][0] == 'Phases':
1412            self.names.insert(-1, new_names)
1413        else:
1414            self.names.append(new_names)
1415        if scale is not None:
1416            pwdrdata['Sample Parameters']['Scale'][0] = scale
1417        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1418            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1419        self.data[histname] = pwdrdata
1420        self.update_ids()
1421
1422        for phase in phases:
1423            phase = self.phase(phase)
1424            self.link_histogram_phase(histname, phase)
1425
1426        return self.histogram(histname)
1427   
1428    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1429        """Loads a phase into the project from a .cif file
1430
1431        :param str phasefile: The CIF file from which to import the phase.
1432        :param str phasename: The name of the new phase, or None for the default
1433        :param list histograms: The names of the histograms to associate with
1434            this phase. Use proj.Histograms() to add to all histograms.
1435        :param str fmthint: If specified, only importers where the format name
1436          (reader.formatName, as shown in Import menu) contains the
1437          supplied string will be tried as importers. If not specified, all
1438          importers consistent with the file extension will be tried
1439          (equivalent to "guess format" in menu).
1440
1441        :returns: A :class:`G2Phase` object representing the
1442            new phase.
1443        """
1444        LoadG2fil()
1445        histograms = [self.histogram(h).name for h in histograms]
1446        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1447
1448        # TODO handle multiple phases in a file
1449        phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
1450        phasereader = phasereaders[0]
1451       
1452        phasename = phasename or phasereader.Phase['General']['Name']
1453        phaseNameList = [p.name for p in self.phases()]
1454        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1455        phasereader.Phase['General']['Name'] = phasename
1456
1457        if 'Phases' not in self.data:
1458            self.data[u'Phases'] = { 'data': None }
1459        assert phasename not in self.data['Phases'], "phase names should be unique"
1460        self.data['Phases'][phasename] = phasereader.Phase
1461
1462        if phasereader.Constraints:
1463            Constraints = self.data['Constraints']
1464            for i in phasereader.Constraints:
1465                if isinstance(i, dict):
1466                    if '_Explain' not in Constraints:
1467                        Constraints['_Explain'] = {}
1468                    Constraints['_Explain'].update(i)
1469                else:
1470                    Constraints['Phase'].append(i)
1471
1472        data = self.data['Phases'][phasename]
1473        generalData = data['General']
1474        SGData = generalData['SGData']
1475        NShkl = len(G2spc.MustrainNames(SGData))
1476        NDij = len(G2spc.HStrainNames(SGData))
1477        Super = generalData.get('Super', 0)
1478        if Super:
1479            SuperVec = np.array(generalData['SuperVec'][0])
1480        else:
1481            SuperVec = []
1482        UseList = data['Histograms']
1483
1484        for hist in histograms:
1485            self.link_histogram_phase(hist, phasename)
1486
1487        for obj in self.names:
1488            if obj[0] == 'Phases':
1489                phasenames = obj
1490                break
1491        else:
1492            phasenames = [u'Phases']
1493            self.names.append(phasenames)
1494        phasenames.append(phasename)
1495
1496        # TODO should it be self.filename, not phasefile?
1497        SetupGeneral(data, os.path.dirname(phasefile))
1498        self.index_ids()
1499
1500        self.update_ids()
1501        return self.phase(phasename)
1502
1503    def link_histogram_phase(self, histogram, phase):
1504        """Associates a given histogram and phase.
1505
1506        .. seealso::
1507
1508            :meth:`G2Project.histogram`
1509            :meth:`G2Project.phase`"""
1510        hist = self.histogram(histogram)
1511        phase = self.phase(phase)
1512
1513        generalData = phase['General']
1514
1515        if hist.name.startswith('HKLF '):
1516            raise NotImplementedError("HKLF not yet supported")
1517        elif hist.name.startswith('PWDR '):
1518            hist['Reflection Lists'][generalData['Name']] = {}
1519            UseList = phase['Histograms']
1520            SGData = generalData['SGData']
1521            NShkl = len(G2spc.MustrainNames(SGData))
1522            NDij = len(G2spc.HStrainNames(SGData))
1523            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1524            UseList[hist.name]['hId'] = hist.id
1525            for key, val in [('Use', True), ('LeBail', False),
1526                             ('newLeBail', True),
1527                             ('Babinet', {'BabA': [0.0, False],
1528                                          'BabU': [0.0, False]})]:
1529                if key not in UseList[hist.name]:
1530                    UseList[hist.name][key] = val
1531        else:
1532            raise RuntimeError("Unexpected histogram" + hist.name)
1533
1534
1535    def reload(self):
1536        """Reload self from self.filename"""
1537        data, names = LoadDictFromProjFile(self.filename)
1538        self.names = names
1539        # Need to deep copy the new data file data into the current tree,
1540        # so that any existing G2Phase, or G2PwdrData objects will still be
1541        # valid
1542        _deep_copy_into(from_=data, into=self.data)
1543
1544    def refine(self, newfile=None, printFile=None, makeBack=False):
1545        # TODO migrate to RefineCore
1546        # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1547        #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1548        # index_ids will automatically save the project
1549        self.index_ids()
1550        # TODO G2strMain does not properly use printFile
1551        G2strMain.Refine(self.filename, makeBack=makeBack)
1552        # Reload yourself
1553        self.reload()
1554
1555    def histogram(self, histname):
1556        """Returns the histogram named histname, or None if it does not exist.
1557
1558        :param histname: The name of the histogram (str), or ranId or index.
1559        :returns: A :class:`G2PwdrData` object, or None if
1560            the histogram does not exist
1561
1562        .. seealso::
1563            :meth:`G2Project.histograms`
1564            :meth:`G2Project.phase`
1565            :meth:`G2Project.phases`
1566            """
1567        if isinstance(histname, G2PwdrData):
1568            if histname.proj == self:
1569                return histname
1570        if histname in self.data:
1571            return G2PwdrData(self.data[histname], self)
1572        try:
1573            # see if histname is an id or ranId
1574            histname = int(histname)
1575        except ValueError:
1576            return
1577
1578        for histogram in self.histograms():
1579            if histogram.id == histname or histogram.ranId == histname:
1580                return histogram
1581
1582    def histograms(self):
1583        """Return a list of all histograms, as
1584        :class:`G2PwdrData` objects
1585
1586        .. seealso::
1587            :meth:`G2Project.histograms`
1588            :meth:`G2Project.phase`
1589            :meth:`G2Project.phases`
1590            """
1591        output = []
1592        # loop through each tree entry. If it is more than one level (more than one item in the
1593        # list of names) then it must be a histogram, unless labeled Phases or Restraints
1594        for obj in self.names:
1595            if len(obj) > 1 and obj[0] != u'Phases' and obj[0] != u'Restraints':
1596                output.append(self.histogram(obj[0]))
1597        return output
1598
1599    def phase(self, phasename):
1600        """
1601        Gives an object representing the specified phase in this project.
1602
1603        :param str phasename: A reference to the desired phase. Either the phase
1604            name (str), the phase's ranId, the phase's index (both int) or
1605            a phase object (:class:`G2Phase`)
1606        :returns: A :class:`G2Phase` object
1607        :raises: KeyError
1608
1609        .. seealso::
1610            :meth:`G2Project.histograms`
1611            :meth:`G2Project.phase`
1612            :meth:`G2Project.phases`
1613            """
1614        if isinstance(phasename, G2Phase):
1615            if phasename.proj == self:
1616                return phasename
1617        phases = self.data['Phases']
1618        if phasename in phases:
1619            return G2Phase(phases[phasename], phasename, self)
1620
1621        try:
1622            # phasename should be phase index or ranId
1623            phasename = int(phasename)
1624        except ValueError:
1625            return
1626
1627        for phase in self.phases():
1628            if phase.id == phasename or phase.ranId == phasename:
1629                return phase
1630
1631    def phases(self):
1632        """
1633        Returns a list of all the phases in the project.
1634
1635        :returns: A list of :class:`G2Phase` objects
1636
1637        .. seealso::
1638            :meth:`G2Project.histogram`
1639            :meth:`G2Project.histograms`
1640            :meth:`G2Project.phase`
1641            """
1642        for obj in self.names:
1643            if obj[0] == 'Phases':
1644                return [self.phase(p) for p in obj[1:]]
1645        return []
1646
1647    def update_ids(self):
1648        """Makes sure all phases and histograms have proper hId and pId"""
1649        # Translated from GetUsedHistogramsAndPhasesfromTree,
1650        #   GSASIIdataGUI.py:4107
1651        for i, h in enumerate(self.histograms()):
1652            h.id = i
1653        for i, p in enumerate(self.phases()):
1654            p.id = i
1655
1656    def do_refinements(self, refinements, histogram='all', phase='all',
1657                       outputnames=None, makeBack=False):
1658        """Conducts one or a series of refinements according to the
1659           input provided in parameter refinements. This is a wrapper
1660           around :meth:`iter_refinements`
1661
1662        :param list refinements: A list of dictionaries specifiying changes to be made to
1663            parameters before refinements are conducted.
1664            See the :ref:`Refinement_recipe` section for how this is defined.
1665        :param str histogram: Name of histogram for refinements to be applied
1666            to, or 'all'; note that this can be overridden for each refinement
1667            step via a "histograms" entry in the dict.
1668        :param str phase: Name of phase for refinements to be applied to, or
1669            'all'; note that this can be overridden for each refinement
1670            step via a "phases" entry in the dict.
1671        :param list outputnames: Provides a list of project (.gpx) file names
1672            to use for each refinement step (specifying None skips the save step).
1673            See :meth:`save`.
1674            Note that this can be overridden using an "output" entry in the dict.
1675        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
1676            before a refinement is performed. The default is False.
1677           
1678        To perform a single refinement without changing any parameters, use this
1679        call:
1680
1681        .. code-block::  python
1682       
1683            my_project.do_refinements([])
1684        """
1685       
1686        for proj in self.iter_refinements(refinements, histogram, phase,
1687                                          outputnames, makeBack):
1688            pass
1689        return self
1690
1691    def iter_refinements(self, refinements, histogram='all', phase='all',
1692                         outputnames=None, makeBack=False):
1693        """Conducts a series of refinements, iteratively. Stops after every
1694        refinement and yields this project, to allow error checking or
1695        logging of intermediate results. Parameter use is the same as for
1696        :meth:`do_refinements` (which calls this method).
1697
1698        >>> def checked_refinements(proj):
1699        ...     for p in proj.iter_refinements(refs):
1700        ...         # Track intermediate results
1701        ...         log(p.histogram('0').residuals)
1702        ...         log(p.phase('0').get_cell())
1703        ...         # Check if parameter diverged, nonsense answer, or whatever
1704        ...         if is_something_wrong(p):
1705        ...             raise Exception("I need a human!")
1706
1707           
1708        """
1709        if outputnames:
1710            if len(refinements) != len(outputnames):
1711                raise ValueError("Should have same number of outputs to"
1712                                 "refinements")
1713        else:
1714            outputnames = [None for r in refinements]
1715
1716        for output, refinedict in zip(outputnames, refinements):
1717            if 'histograms' in refinedict:
1718                hist = refinedict['histograms']
1719            else:
1720                hist = histogram
1721            if 'phases' in refinedict:
1722                ph = refinedict['phases']
1723            else:
1724                ph = phase
1725            if 'output' in refinedict:
1726                output = refinedict['output']
1727            self.set_refinement(refinedict, hist, ph)
1728            # Handle 'once' args - refinements that are disabled after this
1729            # refinement
1730            if 'once' in refinedict:
1731                temp = {'set': refinedict['once']}
1732                self.set_refinement(temp, hist, ph)
1733
1734            if output:
1735                self.save(output)
1736
1737            if 'skip' not in refinedict:
1738                self.refine(makeBack=makeBack)
1739            yield self
1740
1741            # Handle 'once' args - refinements that are disabled after this
1742            # refinement
1743            if 'once' in refinedict:
1744                temp = {'clear': refinedict['once']}
1745                self.set_refinement(temp, hist, ph)
1746            if 'call' in refinedict:
1747                fxn = refinedict['call']
1748                if callable(fxn):
1749                    fxn(*refinedict.get('callargs',[self]))
1750                elif callable(eval(fxn)):
1751                    eval(fxn)(*refinedict.get('callargs',[self]))
1752                else:
1753                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
1754
1755    def set_refinement(self, refinement, histogram='all', phase='all'):
1756        """Apply specified refinements to a given histogram(s) or phase(s).
1757
1758        :param dict refinement: The refinements to be conducted
1759        :param histogram: Specifies either 'all' (default), a single histogram or
1760          a list of histograms. Histograms may be specified as histogram objects
1761          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
1762          of the histogram in the project, numbered starting from 0.
1763          Omitting the parameter or the string 'all' indicates that parameters in
1764          all histograms should be set.
1765        :param phase: Specifies either 'all' (default), a single phase or
1766          a list of phases. Phases may be specified as phase objects
1767          (see :class:`G2Phase`), the phase name (str) or the index number (int)
1768          of the phase in the project, numbered starting from 0.
1769          Omitting the parameter or the string 'all' indicates that parameters in
1770          all phases should be set.
1771
1772        Note that refinement parameters are categorized as one of three types:
1773
1774        1. Histogram parameters
1775        2. Phase parameters
1776        3. Histogram-and-Phase (HAP) parameters
1777       
1778        .. seealso::
1779            :meth:`G2PwdrData.set_refinements`
1780            :meth:`G2PwdrData.clear_refinements`
1781            :meth:`G2Phase.set_refinements`
1782            :meth:`G2Phase.clear_refinements`
1783            :meth:`G2Phase.set_HAP_refinements`
1784            :meth:`G2Phase.clear_HAP_refinements`"""
1785
1786        if histogram == 'all':
1787            hists = self.histograms()
1788        elif isinstance(histogram, list) or isinstance(histogram, tuple):
1789            hists = []
1790            for h in histogram:
1791                if isinstance(h, str) or isinstance(h, int):
1792                    hists.append(self.histogram(h))
1793                else:
1794                    hists.append(h)
1795        elif isinstance(histogram, str) or isinstance(histogram, int):
1796            hists = [self.histogram(histogram)]
1797        else:
1798            hists = [histogram]
1799
1800        if phase == 'all':
1801            phases = self.phases()
1802        elif isinstance(phase, list) or isinstance(phase, tuple):
1803            phases = []
1804            for ph in phase:
1805                if isinstance(ph, str) or isinstance(ph, int):
1806                    phases.append(self.phase(ph))
1807                else:
1808                    phases.append(ph)
1809        elif isinstance(phase, str) or isinstance(phase, int):
1810            phases = [self.phase(phase)]
1811        else:
1812            phases = [phase]
1813
1814        # TODO: HAP parameters:
1815        #   Babinet
1816        #   Extinction
1817        #   HStrain
1818        #   Mustrain
1819        #   Pref. Ori
1820        #   Size
1821
1822        pwdr_set = {}
1823        phase_set = {}
1824        hap_set = {}
1825        for key, val in refinement.get('set', {}).items():
1826            # Apply refinement options
1827            if G2PwdrData.is_valid_refinement_key(key):
1828                pwdr_set[key] = val
1829            elif G2Phase.is_valid_refinement_key(key):
1830                phase_set[key] = val
1831            elif G2Phase.is_valid_HAP_refinement_key(key):
1832                hap_set[key] = val
1833            else:
1834                raise ValueError("Unknown refinement key", key)
1835
1836        for hist in hists:
1837            hist.set_refinements(pwdr_set)
1838        for phase in phases:
1839            phase.set_refinements(phase_set)
1840        for phase in phases:
1841            phase.set_HAP_refinements(hap_set, hists)
1842
1843        pwdr_clear = {}
1844        phase_clear = {}
1845        hap_clear = {}
1846        for key, val in refinement.get('clear', {}).items():
1847            # Clear refinement options
1848            if G2PwdrData.is_valid_refinement_key(key):
1849                pwdr_clear[key] = val
1850            elif G2Phase.is_valid_refinement_key(key):
1851                phase_clear[key] = val
1852            elif G2Phase.is_valid_HAP_refinement_key(key):
1853                hap_set[key] = val
1854            else:
1855                raise ValueError("Unknown refinement key", key)
1856
1857        for hist in hists:
1858            hist.clear_refinements(pwdr_clear)
1859        for phase in phases:
1860            phase.clear_refinements(phase_clear)
1861        for phase in phases:
1862            phase.clear_HAP_refinements(hap_clear, hists)
1863
1864    def index_ids(self):
1865        import GSASIIstrIO as G2strIO
1866        self.save()
1867        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
1868
1869    def add_constraint_raw(self, cons_scope, constr):
1870        """Adds a constraint of type consType to the project.
1871        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
1872
1873        WARNING it does not check the constraint is well-constructed"""
1874        constrs = self.data['Constraints']['data']
1875        if 'Global' not in constrs:
1876            constrs['Global'] = []
1877        constrs[cons_scope].append(constr)
1878
1879    def hold_many(self, vars, type):
1880        """Apply holds for all the variables in vars, for constraint of a given type.
1881
1882        type is passed directly to add_constraint_raw as consType
1883
1884        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
1885            string variable specifiers, or arguments for :meth:`make_var_obj`
1886        :param str type: A string constraint type specifier. See
1887            :class:`G2Project.add_constraint_raw`
1888
1889        """
1890        for var in vars:
1891            if isinstance(var, str):
1892                var = self.make_var_obj(var)
1893            elif not isinstance(var, G2obj.G2VarObj):
1894                var = self.make_var_obj(*var)
1895            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
1896
1897    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
1898                     reloadIdx=True):
1899        """Wrapper to create a G2VarObj. Takes either a string representaiton ("p:h:name:a")
1900        or individual names of phase, histogram, varname, and atomId.
1901
1902        Automatically converts string phase, hist, or atom names into the ID required
1903        by G2VarObj."""
1904
1905        if reloadIdx:
1906            self.index_ids()
1907
1908        # If string representation, short circuit
1909        if hist is None and varname is None and atomId is None:
1910            if isinstance(phase, str) and ':' in phase:
1911                return G2obj.G2VarObj(phase)
1912
1913        # Get phase index
1914        phaseObj = None
1915        if isinstance(phase, G2Phase):
1916            phaseObj = phase
1917            phase = G2obj.PhaseRanIdLookup[phase.ranId]
1918        elif phase in self.data['Phases']:
1919            phaseObj = self.phase(phase)
1920            phaseRanId = phaseObj.ranId
1921            phase = G2obj.PhaseRanIdLookup[phaseRanId]
1922
1923        # Get histogram index
1924        if isinstance(hist, G2PwdrData):
1925            hist = G2obj.HistRanIdLookup[hist.ranId]
1926        elif hist in self.data:
1927            histRanId = self.histogram(hist).ranId
1928            hist = G2obj.HistRanIdLookup[histRanId]
1929
1930        # Get atom index (if any)
1931        if isinstance(atomId, G2AtomRecord):
1932            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
1933        elif phaseObj:
1934            atomObj = phaseObj.atom(atomId)
1935            if atomObj:
1936                atomRanId = atomObj.ranId
1937                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
1938
1939        return G2obj.G2VarObj(phase, hist, varname, atomId)
1940
1941
1942class G2AtomRecord(G2ObjectWrapper):
1943    """Wrapper for an atom record. Has convenient accessors via @property.
1944
1945
1946    Available accessors: label, type, refinement_flags, coordinates,
1947        occupancy, ranId, id, adp_flag, uiso
1948
1949    Example:
1950
1951    >>> atom = some_phase.atom("O3")
1952    >>> # We can access the underlying data format
1953    >>> atom.data
1954    ['O3', 'O-2', '', ... ]
1955    >>> # We can also use wrapper accessors
1956    >>> atom.coordinates
1957    (0.33, 0.15, 0.5)
1958    >>> atom.refinement_flags
1959    u'FX'
1960    >>> atom.ranId
1961    4615973324315876477
1962    >>> atom.occupancy
1963    1.0
1964    """
1965    def __init__(self, data, indices, proj):
1966        self.data = data
1967        self.cx, self.ct, self.cs, self.cia = indices
1968        self.proj = proj
1969
1970    @property
1971    def label(self):
1972        return self.data[self.ct-1]
1973
1974    @property
1975    def type(self):
1976        return self.data[self.ct]
1977
1978    @property
1979    def refinement_flags(self):
1980        return self.data[self.ct+1]
1981
1982    @refinement_flags.setter
1983    def refinement_flags(self, other):
1984        # Automatically check it is a valid refinement
1985        for c in other:
1986            if c not in ' FXU':
1987                raise ValueError("Invalid atom refinement: ", other)
1988        self.data[self.ct+1] = other
1989
1990    @property
1991    def coordinates(self):
1992        return tuple(self.data[self.cx:self.cx+3])
1993
1994    @property
1995    def occupancy(self):
1996        return self.data[self.cx+3]
1997
1998    @occupancy.setter
1999    def occupancy(self, val):
2000        self.data[self.cx+3] = float(val)
2001
2002    @property
2003    def ranId(self):
2004        return self.data[self.cia+8]
2005
2006    @property
2007    def adp_flag(self):
2008        # Either 'I' or 'A'
2009        return self.data[self.cia]
2010
2011    @property
2012    def uiso(self):
2013        if self.adp_flag == 'I':
2014            return self.data[self.cia+1]
2015        else:
2016            return self.data[self.cia+2:self.cia+8]
2017
2018    @uiso.setter
2019    def uiso(self, value):
2020        if self.adp_flag == 'I':
2021            self.data[self.cia+1] = float(value)
2022        else:
2023            assert len(value) == 6
2024            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2025
2026
2027class G2PwdrData(G2ObjectWrapper):
2028    """Wraps a Powder Data Histogram."""
2029    def __init__(self, data, proj):
2030        self.data = data
2031        self.proj = proj
2032
2033    @staticmethod
2034    def is_valid_refinement_key(key):
2035        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2036                      'Instrument Parameters']
2037        return key in valid_keys
2038
2039    @property
2040    def name(self):
2041        return self.data['data'][-1]
2042
2043    @property
2044    def ranId(self):
2045        return self.data['data'][0]['ranId']
2046
2047    @property
2048    def residuals(self):
2049        '''Provides a dictionary with with the R-factors for this histogram.
2050        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2051        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2052        '''
2053        data = self.data['data'][0]
2054        return {key: data[key] for key in data
2055                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2056                   or ':' in key}
2057
2058    @property
2059    def InstrumentParameters(self):
2060        '''Provides a dictionary with with the Instrument Parameters
2061        for this histogram.
2062        '''
2063        return self.data['Instrument Parameters'][0]
2064
2065    @property
2066    def SampleParameters(self):
2067        '''Provides a dictionary with with the Sample Parameters
2068        for this histogram.
2069        '''
2070        return self.data['Sample Parameters']
2071   
2072    @property
2073    def id(self):
2074        self.proj.update_ids()
2075        return self.data['data'][0]['hId']
2076
2077    @id.setter
2078    def id(self, val):
2079        self.data['data'][0]['hId'] = val
2080
2081    def fit_fixed_points(self):
2082        """Attempts to apply a background fit to the fixed points currently specified."""
2083        def SetInstParms(Inst):
2084            dataType = Inst['Type'][0]
2085            insVary = []
2086            insNames = []
2087            insVals = []
2088            for parm in Inst:
2089                insNames.append(parm)
2090                insVals.append(Inst[parm][1])
2091                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2092                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2093                        Inst[parm][2] = False
2094            instDict = dict(zip(insNames, insVals))
2095            instDict['X'] = max(instDict['X'], 0.01)
2096            instDict['Y'] = max(instDict['Y'], 0.01)
2097            if 'SH/L' in instDict:
2098                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2099            return dataType, instDict, insVary
2100
2101        bgrnd = self.data['Background']
2102
2103        # Need our fixed points in order
2104        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2105        X = [x for x, y in bgrnd[1]['FixedPoints']]
2106        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2107
2108        limits = self.data['Limits'][1]
2109        if X[0] > limits[0]:
2110            X = [limits[0]] + X
2111            Y = [Y[0]] + Y
2112        if X[-1] < limits[1]:
2113            X += [limits[1]]
2114            Y += [Y[-1]]
2115
2116        # Some simple lookups
2117        controls = self.proj['Controls']['data']
2118        inst, inst2 = self.data['Instrument Parameters']
2119        pwddata = self.data['data'][1]
2120
2121        # Construct the data for background fitting
2122        xBeg = np.searchsorted(pwddata[0], limits[0])
2123        xFin = np.searchsorted(pwddata[0], limits[1])
2124        xdata = pwddata[0][xBeg:xFin]
2125        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2126
2127        W = [1]*len(xdata)
2128        Z = [0]*len(xdata)
2129
2130        dataType, insDict, insVary = SetInstParms(inst)
2131        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2132
2133        # Do the fit
2134        data = np.array([xdata, ydata, W, Z, Z, Z])
2135        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2136                        prevVaryList=bakVary, controls=controls)
2137
2138        # Post-fit
2139        parmDict = {}
2140        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2141        parmDict.update(bakDict)
2142        parmDict.update(insDict)
2143        pwddata[3][xBeg:xFin] *= 0
2144        pwddata[5][xBeg:xFin] *= 0
2145        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2146
2147        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2148        # TODO update background
2149        self.proj.save()
2150
2151    def getdata(self,datatype):
2152        '''Provides access to the histogram data of the selected data type
2153
2154        :param str datatype: must be one of the following values (case is ignored)
2155       
2156           * 'X': the 2theta or TOF values for the pattern
2157           * 'Yobs': the observed intensity values
2158           * 'Yweight': the weights for each data point (1/sigma**2)
2159           * 'Ycalc': the computed intensity values
2160           * 'Background': the computed background values
2161           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2162
2163        :returns: an numpy MaskedArray with data values of the requested type
2164       
2165        '''
2166        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2167        if datatype.lower() not in enums:
2168            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2169        return self.data['data'][1][enums.index(datatype.lower())]
2170       
2171    def y_calc(self):
2172        return self.data['data'][1][3]
2173
2174    def Export(self,fileroot,extension):
2175        '''Write the histogram into a file. The path is specified by fileroot and
2176        extension.
2177       
2178        :param str fileroot: name of the file, optionally with a path (extension is
2179           ignored)
2180        :param str extension: includes '.', must match an extension in global
2181           exportersByExtension['powder'] or a Exception is raised.
2182        :returns: name of file that was written
2183        '''
2184        if extension not in exportersByExtension.get('powder',[]):
2185            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2186        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2187        obj = exportersByExtension['powder'][extension]
2188        obj.SetFromArray(hist=self.data,histname=self.name)
2189        obj.Writer(self.name,fil)
2190           
2191    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2192        try:
2193            import matplotlib.pyplot as plt
2194            data = self.data['data'][1]
2195            if Yobs:
2196                plt.plot(data[0], data[1], label='Yobs')
2197            if Ycalc:
2198                plt.plot(data[0], data[3], label='Ycalc')
2199            if Background:
2200                plt.plot(data[0], data[4], label='Background')
2201            if Residual:
2202                plt.plot(data[0], data[5], label="Residual")
2203        except ImportError:
2204            pass
2205
2206    def get_wR(self):
2207        """returns the overall weighted profile R factor for a histogram
2208       
2209        :returns: a wR value as a percentage or None if not defined
2210        """
2211        return self['data'][0].get('wR')
2212
2213    def set_refinements(self, refs):
2214        """Sets the refinement parameter 'key' to the specification 'value'
2215
2216        :param dict refs: A dictionary of the parameters to be set. See
2217                          :ref:`Histogram_parameters_table` for a description of
2218                          what these dictionaries should be.
2219
2220        :returns: None
2221
2222        """
2223        do_fit_fixed_points = False
2224        for key, value in refs.items():
2225            if key == 'Limits':
2226                old_limits = self.data['Limits'][1]
2227                new_limits = value
2228                if isinstance(new_limits, dict):
2229                    if 'low' in new_limits:
2230                        old_limits[0] = new_limits['low']
2231                    if 'high' in new_limits:
2232                        old_limits[1] = new_limits['high']
2233                else:
2234                    old_limits[0], old_limits[1] = new_limits
2235            elif key == 'Sample Parameters':
2236                sample = self.data['Sample Parameters']
2237                for sparam in value:
2238                    if sparam not in sample:
2239                        raise ValueError("Unknown refinement parameter, "
2240                                         + str(sparam))
2241                    sample[sparam][1] = True
2242            elif key == 'Background':
2243                bkg, peaks = self.data['Background']
2244
2245                # If True or False, just set the refine parameter
2246                if value in (True, False):
2247                    bkg[1] = value
2248                    return
2249
2250                if 'type' in value:
2251                    bkg[0] = value['type']
2252                if 'refine' in value:
2253                    bkg[1] = value['refine']
2254                if 'no. coeffs' in value:
2255                    cur_coeffs = bkg[2]
2256                    n_coeffs = value['no. coeffs']
2257                    if n_coeffs > cur_coeffs:
2258                        for x in range(n_coeffs - cur_coeffs):
2259                            bkg.append(0.0)
2260                    else:
2261                        for _ in range(cur_coeffs - n_coeffs):
2262                            bkg.pop()
2263                    bkg[2] = n_coeffs
2264                if 'coeffs' in value:
2265                    bkg[3:] = value['coeffs']
2266                if 'FixedPoints' in value:
2267                    peaks['FixedPoints'] = [(float(a), float(b))
2268                                            for a, b in value['FixedPoints']]
2269                if value.get('fit fixed points', False):
2270                    do_fit_fixed_points = True
2271
2272            elif key == 'Instrument Parameters':
2273                instrument, secondary = self.data['Instrument Parameters']
2274                for iparam in value:
2275                    try:
2276                        instrument[iparam][2] = True
2277                    except IndexError:
2278                        raise ValueError("Invalid key:", iparam)
2279            else:
2280                raise ValueError("Unknown key:", key)
2281        # Fit fixed points after the fact - ensure they are after fixed points
2282        # are added
2283        if do_fit_fixed_points:
2284            # Background won't be fit if refinement flag not set
2285            orig = self.data['Background'][0][1]
2286            self.data['Background'][0][1] = True
2287            self.fit_fixed_points()
2288            # Restore the previous value
2289            self.data['Background'][0][1] = orig
2290
2291    def clear_refinements(self, refs):
2292        """Clears the refinement parameter 'key' and its associated value.
2293
2294        :param dict refs: A dictionary of parameters to clear."""
2295        for key, value in refs.items():
2296            if key == 'Limits':
2297                old_limits, cur_limits = self.data['Limits']
2298                cur_limits[0], cur_limits[1] = old_limits
2299            elif key == 'Sample Parameters':
2300                sample = self.data['Sample Parameters']
2301                for sparam in value:
2302                    sample[sparam][1] = False
2303            elif key == 'Background':
2304                bkg, peaks = self.data['Background']
2305
2306                # If True or False, just set the refine parameter
2307                if value in (True, False):
2308                    bkg[1] = False
2309                    return
2310
2311                bkg[1] = False
2312                if 'FixedPoints' in value:
2313                    if 'FixedPoints' in peaks:
2314                        del peaks['FixedPoints']
2315            elif key == 'Instrument Parameters':
2316                instrument, secondary = self.data['Instrument Parameters']
2317                for iparam in value:
2318                    instrument[iparam][2] = False
2319            else:
2320                raise ValueError("Unknown key:", key)
2321
2322    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2323        '''Adds a single peak to the peak list
2324        :param float area: peak area
2325        :param float dspace: peak position as d-space (A)
2326        :param float Q: peak position as Q (A-1)
2327        :param float ttheta: peak position as 2Theta (deg)
2328
2329        Note: only one of the parameters dspace, Q or ttheta may be specified
2330        '''
2331        import GSASIIlattice as G2lat
2332        import GSASIImath as G2mth
2333        if (not dspace) + (not Q) + (not ttheta) != 2:
2334            print('add_peak error: too many or no peak position(s) specified')
2335            return
2336        pos = ttheta
2337        Parms,Parms2 = self.data['Instrument Parameters']
2338        if Q:
2339            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2340        elif dspace:
2341            pos = G2lat.Dsp2pos(Parms,dspace)
2342        peaks = self.data['Peak List']
2343        peaks['sigDict'] = {}        #no longer valid
2344        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2345
2346    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2347        '''Set refinement flags for peaks
2348       
2349        :param list peaklist: a list of peaks to change flags. If None (default), changes
2350          are made to all peaks.
2351        :param bool area: Sets or clears the refinement flag for the peak area value.
2352          If None (the default), no change is made.
2353        :param bool pos: Sets or clears the refinement flag for the peak position value.
2354          If None (the default), no change is made.
2355        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2356          If None (the default), no change is made.
2357        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2358          If None (the default), no change is made.
2359         
2360        Note that when peaks are first created the area flag is on and the other flags are
2361        initially off.
2362
2363        Example::
2364       
2365           set_peakFlags(sig=False,gam=True)
2366
2367        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
2368        all peaks. The position and area flags are not changed from their previous values.
2369        '''
2370        peaks = self.data['Peak List']
2371        if peaklist is None:
2372            peaklist = range(len(peaks['peaks']))
2373        for i in peaklist:
2374            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
2375                if var is not None:
2376                    peaks['peaks'][i][j] = var
2377           
2378    def refine_peaks(self):
2379        '''Causes a refinement of peak position, background and instrument parameters
2380        '''
2381        import GSASIIpwd as G2pwd
2382        controls = self.proj.data.get('Controls',{})
2383        controls = controls.get('data',
2384                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
2385                            )
2386        peaks = self.data['Peak List']
2387        Parms,Parms2 = self.data['Instrument Parameters']
2388        background = self.data['Background']
2389        limits = self.data['Limits'][1]
2390        bxye = np.zeros(len(self.data['data'][1][1]))
2391        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
2392                                           Parms,Parms2,self.data['data'][1],bxye,[],
2393                                           False,controls,None)[0]
2394
2395    def SaveProfile(self,filename):
2396        '''Writes a GSAS-II (new style) .instprm file
2397        '''
2398        data,Parms2 = self.data['Instrument Parameters']
2399        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2400        File = open(filename,'w')
2401        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
2402        for item in data:
2403            File.write(item+':'+str(data[item][1])+'\n')
2404        File.close()
2405        print ('Instrument parameters saved to: '+filename)
2406
2407    def LoadProfile(self,filename):
2408        '''Reads a GSAS-II (new style) .instprm file and overwrites the current parameters
2409        '''
2410        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2411        File = open(filename,'r')
2412        S = File.readline()
2413        newItems = []
2414        newVals = []
2415        Found = False
2416        while S:
2417            if S[0] == '#':
2418                if Found:
2419                    break
2420                if 'Bank' in S:
2421                    if bank == int(S.split(':')[0].split()[1]):
2422                        S = File.readline()
2423                        continue
2424                    else:
2425                        S = File.readline()
2426                        while S and '#Bank' not in S:
2427                            S = File.readline()
2428                        continue
2429                else:   #a non #Bank file
2430                    S = File.readline()
2431                    continue
2432            Found = True
2433            [item,val] = S[:-1].split(':')
2434            newItems.append(item)
2435            try:
2436                newVals.append(float(val))
2437            except ValueError:
2438                newVals.append(val)                       
2439            S = File.readline()               
2440        File.close()
2441        LoadG2fil()
2442        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
2443
2444       
2445class G2Phase(G2ObjectWrapper):
2446    """A wrapper object around a given phase.
2447
2448    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
2449    """
2450    def __init__(self, data, name, proj):
2451        self.data = data
2452        self.name = name
2453        self.proj = proj
2454
2455    @staticmethod
2456    def is_valid_refinement_key(key):
2457        valid_keys = ["Cell", "Atoms", "LeBail"]
2458        return key in valid_keys
2459
2460    @staticmethod
2461    def is_valid_HAP_refinement_key(key):
2462        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
2463                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
2464        return key in valid_keys
2465
2466    def atom(self, atomlabel):
2467        """Returns the atom specified by atomlabel, or None if it does not
2468        exist.
2469
2470        :param str atomlabel: The name of the atom (e.g. "O2")
2471        :returns: A :class:`G2AtomRecord` object
2472            representing the atom.
2473        """
2474        # Consult GSASIIobj.py for the meaning of this
2475        cx, ct, cs, cia = self.data['General']['AtomPtrs']
2476        ptrs = [cx, ct, cs, cia]
2477        atoms = self.data['Atoms']
2478        for atom in atoms:
2479            if atom[ct-1] == atomlabel:
2480                return G2AtomRecord(atom, ptrs, self.proj)
2481
2482    def atoms(self):
2483        """Returns a list of atoms present in the phase.
2484
2485        :returns: A list of :class:`G2AtomRecord` objects.
2486
2487        .. seealso::
2488            :meth:`G2Phase.atom`
2489            :class:`G2AtomRecord`
2490        """
2491        ptrs = self.data['General']['AtomPtrs']
2492        output = []
2493        atoms = self.data['Atoms']
2494        for atom in atoms:
2495            output.append(G2AtomRecord(atom, ptrs, self.proj))
2496        return output
2497
2498    def histograms(self):
2499        output = []
2500        for hname in self.data.get('Histograms', {}).keys():
2501            output.append(self.proj.histogram(hname))
2502        return output
2503
2504    @property
2505    def ranId(self):
2506        return self.data['ranId']
2507
2508    @property
2509    def id(self):
2510        return self.data['pId']
2511
2512    @id.setter
2513    def id(self, val):
2514        self.data['pId'] = val
2515
2516    def get_cell(self):
2517        """Returns a dictionary of the cell parameters, with keys:
2518            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
2519
2520        :returns: a dict
2521
2522        .. seealso::
2523           :meth:`G2Phase.get_cell_and_esd`
2524
2525        """
2526        cell = self.data['General']['Cell']
2527        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
2528                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
2529                'volume': cell[7]}
2530
2531    def get_cell_and_esd(self):
2532        """
2533        Returns a pair of dictionaries, the first representing the unit cell, the second
2534        representing the estimated standard deviations of the unit cell.
2535
2536        :returns: a tuple of two dictionaries
2537
2538        .. seealso::
2539           :meth:`G2Phase.get_cell`
2540
2541        """
2542        # translated from GSASIIstrIO.ExportBaseclass.GetCell
2543        import GSASIIstrIO as G2stIO
2544        import GSASIIlattice as G2lat
2545        import GSASIImapvars as G2mv
2546        try:
2547            pfx = str(self.id) + '::'
2548            sgdata = self['General']['SGData']
2549            covDict = self.proj['Covariance']['data']
2550
2551            parmDict = dict(zip(covDict.get('varyList',[]),
2552                                covDict.get('variables',[])))
2553            sigDict = dict(zip(covDict.get('varyList',[]),
2554                               covDict.get('sig',[])))
2555
2556            if covDict.get('covMatrix') is not None:
2557                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2558                                                  covDict['varyList'],
2559                                                  parmDict))
2560
2561            A, sigA = G2stIO.cellFill(pfx, sgdata, parmDict, sigDict)
2562            cellSig = G2stIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
2563            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
2564            cellDict, cellSigDict = {}, {}
2565            for i, key in enumerate(['length_a', 'length_b', 'length_c',
2566                                     'angle_alpha', 'angle_beta', 'angle_gamma',
2567                                     'volume']):
2568                cellDict[key] = cellList[i]
2569                cellSigDict[key] = cellSig[i]
2570            return cellDict, cellSigDict
2571        except KeyError:
2572            cell = self.get_cell()
2573            return cell, {key: 0.0 for key in cell}
2574
2575    def export_CIF(self, outputname, quickmode=True):
2576        """Write this phase to a .cif file named outputname
2577
2578        :param str outputname: The name of the .cif file to write to
2579        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
2580        # This code is all taken from exports/G2export_CIF.py
2581        # Functions copied have the same names
2582        import GSASIImath as G2mth
2583        import GSASIImapvars as G2mv
2584        from exports import G2export_CIF as cif
2585
2586        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
2587        CIFname = os.path.splitext(self.proj.filename)[0]
2588        CIFname = os.path.split(CIFname)[1]
2589        CIFname = ''.join([c if ord(c) < 128 else ''
2590                           for c in CIFname.replace(' ', '_')])
2591        try:
2592            author = self.proj['Controls']['data'].get('Author','').strip()
2593        except KeyError:
2594            pass
2595        oneblock = True
2596
2597        covDict = self.proj['Covariance']['data']
2598        parmDict = dict(zip(covDict.get('varyList',[]),
2599                            covDict.get('variables',[])))
2600        sigDict = dict(zip(covDict.get('varyList',[]),
2601                           covDict.get('sig',[])))
2602
2603        if covDict.get('covMatrix') is not None:
2604            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2605                                              covDict['varyList'],
2606                                              parmDict))
2607
2608        with open(outputname, 'w') as fp:
2609            fp.write(' \n' + 70*'#' + '\n')
2610            cif.WriteCIFitem(fp, 'data_' + CIFname)
2611            # from exports.G2export_CIF.WritePhaseInfo
2612            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
2613            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
2614            # TODO get esds
2615            cellDict = self.get_cell()
2616            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
2617            names = ['length_a','length_b','length_c',
2618                     'angle_alpha','angle_beta ','angle_gamma',
2619                     'volume']
2620            for key, val in cellDict.items():
2621                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
2622
2623            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
2624                         self.data['General']['SGData']['SGSys'])
2625
2626            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
2627            # regularize capitalization and remove trailing H/R
2628            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
2629            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
2630
2631            # generate symmetry operations including centering and center of symmetry
2632            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
2633                self.data['General']['SGData'])
2634            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
2635            for i, op in enumerate(SymOpList,start=1):
2636                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
2637
2638            # TODO skipped histograms, exports/G2export_CIF.py:880
2639
2640            # report atom params
2641            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
2642                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
2643                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
2644            else:
2645                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
2646            # report cell contents
2647            cif.WriteComposition(fp, self.data, self.name, parmDict)
2648            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
2649                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
2650                raise NotImplementedError("only quickmode currently supported")
2651            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
2652                cif.WriteCIFitem(fp,'\n# Difference density results')
2653                MinMax = self.data['General']['Map']['minmax']
2654                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
2655                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
2656
2657
2658    def set_refinements(self, refs):
2659        """Sets the refinement parameter 'key' to the specification 'value'
2660
2661        :param dict refs: A dictionary of the parameters to be set. See
2662                          :ref:`Phase_parameters_table` for a description of
2663                          this dictionary.
2664
2665        :returns: None"""
2666        for key, value in refs.items():
2667            if key == "Cell":
2668                self.data['General']['Cell'][0] = value
2669
2670            elif key == "Atoms":
2671                for atomlabel, atomrefinement in value.items():
2672                    if atomlabel == 'all':
2673                        for atom in self.atoms():
2674                            atom.refinement_flags = atomrefinement
2675                    else:
2676                        atom = self.atom(atomlabel)
2677                        if atom is None:
2678                            raise ValueError("No such atom: " + atomlabel)
2679                        atom.refinement_flags = atomrefinement
2680
2681            elif key == "LeBail":
2682                hists = self.data['Histograms']
2683                for hname, hoptions in hists.items():
2684                    if 'LeBail' not in hoptions:
2685                        hoptions['newLeBail'] = bool(True)
2686                    hoptions['LeBail'] = bool(value)
2687            else:
2688                raise ValueError("Unknown key:", key)
2689
2690    def clear_refinements(self, refs):
2691        """Clears a given set of parameters.
2692
2693        :param dict refs: The parameters to clear"""
2694        for key, value in refs.items():
2695            if key == "Cell":
2696                self.data['General']['Cell'][0] = False
2697            elif key == "Atoms":
2698                cx, ct, cs, cia = self.data['General']['AtomPtrs']
2699
2700                for atomlabel in value:
2701                    atom = self.atom(atomlabel)
2702                    # Set refinement to none
2703                    atom.refinement_flags = ' '
2704            elif key == "LeBail":
2705                hists = self.data['Histograms']
2706                for hname, hoptions in hists.items():
2707                    if 'LeBail' not in hoptions:
2708                        hoptions['newLeBail'] = True
2709                    hoptions['LeBail'] = False
2710            else:
2711                raise ValueError("Unknown key:", key)
2712
2713    def set_HAP_refinements(self, refs, histograms='all'):
2714        """Sets the given HAP refinement parameters between this phase and
2715        the given histograms
2716
2717        :param dict refs: A dictionary of the parameters to be set. See
2718                          :ref:`HAP_parameters_table` for a description of this
2719                          dictionary.
2720        :param histograms: Either 'all' (default) or a list of the histograms
2721            whose HAP parameters will be set with this phase. Histogram and phase
2722            must already be associated
2723
2724        :returns: None
2725        """
2726        if histograms == 'all':
2727            histograms = self.data['Histograms'].values()
2728        else:
2729            histograms = [self.data['Histograms'][h.name] for h in histograms]
2730
2731        for key, val in refs.items():
2732            for h in histograms:
2733                if key == 'Babinet':
2734                    try:
2735                        sets = list(val)
2736                    except ValueError:
2737                        sets = ['BabA', 'BabU']
2738                    for param in sets:
2739                        if param not in ['BabA', 'BabU']:
2740                            raise ValueError("Not sure what to do with" + param)
2741                        for hist in histograms:
2742                            hist['Babinet'][param][1] = True
2743                elif key == 'Extinction':
2744                    for h in histograms:
2745                        h['Extinction'][1] = bool(val)
2746                elif key == 'HStrain':
2747                    for h in histograms:
2748                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
2749                elif key == 'Mustrain':
2750                    for h in histograms:
2751                        mustrain = h['Mustrain']
2752                        newType = None
2753                        direction = None
2754                        if isinstance(val, strtypes):
2755                            if val in ['isotropic', 'uniaxial', 'generalized']:
2756                                newType = val
2757                            else:
2758                                raise ValueError("Not a Mustrain type: " + val)
2759                        elif isinstance(val, dict):
2760                            newType = val.get('type', None)
2761                            direction = val.get('direction', None)
2762
2763                        if newType:
2764                            mustrain[0] = newType
2765                            if newType == 'isotropic':
2766                                mustrain[2][0] = True == val.get('refine',False)
2767                                mustrain[5] = [False for p in mustrain[4]]
2768                            elif newType == 'uniaxial':
2769                                if 'refine' in val:
2770                                    mustrain[2][0] = False
2771                                    types = val['refine']
2772                                    if isinstance(types, strtypes):
2773                                        types = [types]
2774                                    elif isinstance(types, bool):
2775                                        mustrain[2][1] = types
2776                                        mustrain[2][2] = types
2777                                        types = []
2778                                    else:
2779                                        raise ValueError("Not sure what to do with: "
2780                                                         + str(types))
2781                                else:
2782                                    types = []
2783
2784                                for unitype in types:
2785                                    if unitype == 'equatorial':
2786                                        mustrain[2][0] = True
2787                                    elif unitype == 'axial':
2788                                        mustrain[2][1] = True
2789                                    else:
2790                                        msg = 'Invalid uniaxial mustrain type'
2791                                        raise ValueError(msg + ': ' + unitype)
2792                            else:  # newtype == 'generalized'
2793                                mustrain[2] = [False for p in mustrain[1]]
2794                                if 'refine' in val:
2795                                    mustrain[5] = [True == val['refine']]*len(mustrain[5])
2796
2797                        if direction:
2798                            if len(direction) != 3:
2799                                raise ValueError("Expected hkl, found", direction)
2800                            direction = [int(n) for n in direction]
2801                            mustrain[3] = direction
2802                elif key == 'Size':
2803                    newSize = None
2804                    if 'value' in val:
2805                        newSize = float(val['value'])
2806                    for h in histograms:
2807                        size = h['Size']
2808                        newType = None
2809                        direction = None
2810                        if isinstance(val, strtypes):
2811                            if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
2812                                newType = val
2813                            else:
2814                                raise ValueError("Not a valid Size type: " + val)
2815                        elif isinstance(val, dict):
2816                            newType = val.get('type', None)
2817                            direction = val.get('direction', None)
2818
2819                        if newType:
2820                            size[0] = newType
2821                            refine = True == val.get('refine')
2822                            if newType == 'isotropic' and refine is not None:
2823                                size[2][0] = bool(refine)
2824                                if newSize: size[1][0] = newSize
2825                            elif newType == 'uniaxial' and refine is not None:
2826                                size[2][1] = bool(refine)
2827                                size[2][2] = bool(refine)
2828                                if newSize: size[1][1] = size[1][2] =newSize
2829                            elif newType == 'ellipsoidal' and refine is not None:
2830                                size[5] = [bool(refine) for p in size[5]]
2831                                if newSize: size[4] = [newSize for p in size[4]]
2832
2833                        if direction:
2834                            if len(direction) != 3:
2835                                raise ValueError("Expected hkl, found", direction)
2836                            direction = [int(n) for n in direction]
2837                            size[3] = direction
2838                elif key == 'Pref.Ori.':
2839                    for h in histograms:
2840                        h['Pref.Ori.'][2] = bool(val)
2841                elif key == 'Show':
2842                    for h in histograms:
2843                        h['Show'] = bool(val)
2844                elif key == 'Use':
2845                    for h in histograms:
2846                        h['Use'] = bool(val)
2847                elif key == 'Scale':
2848                    for h in histograms:
2849                        h['Scale'][1] = bool(val)
2850                else:
2851                    print(u'Unknown HAP key: '+key)
2852
2853    def getHAPvalues(self, histname):
2854        """Returns a dict with HAP values for the selected histogram
2855
2856        :param histogram: is a histogram object (:class:`G2PwdrData`) or
2857            a histogram name or the index number of the histogram
2858
2859        :returns: HAP value dict
2860        """
2861        if isinstance(histname, G2PwdrData):
2862            histname = histname.name
2863        elif histname in self.data['Histograms']:
2864            pass
2865        elif type(histname) is int:
2866            histname = self.proj.histograms()[histname].name
2867        else:
2868            raise G2ScriptException("Invalid histogram reference: "+str(histname))
2869        return self.data['Histograms'][histname]
2870                   
2871    def clear_HAP_refinements(self, refs, histograms='all'):
2872        """Clears the given HAP refinement parameters between this phase and
2873        the given histograms
2874
2875        :param dict refs: A dictionary of the parameters to be cleared.
2876        :param histograms: Either 'all' (default) or a list of the histograms
2877            whose HAP parameters will be cleared with this phase. Histogram and
2878            phase must already be associated
2879
2880        :returns: None
2881        """
2882        if histograms == 'all':
2883            histograms = self.data['Histograms'].values()
2884        else:
2885            histograms = [self.data['Histograms'][h.name] for h in histograms]
2886
2887        for key, val in refs.items():
2888            for h in histograms:
2889                if key == 'Babinet':
2890                    try:
2891                        sets = list(val)
2892                    except ValueError:
2893                        sets = ['BabA', 'BabU']
2894                    for param in sets:
2895                        if param not in ['BabA', 'BabU']:
2896                            raise ValueError("Not sure what to do with" + param)
2897                        for hist in histograms:
2898                            hist['Babinet'][param][1] = False
2899                elif key == 'Extinction':
2900                    for h in histograms:
2901                        h['Extinction'][1] = False
2902                elif key == 'HStrain':
2903                    for h in histograms:
2904                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
2905                elif key == 'Mustrain':
2906                    for h in histograms:
2907                        mustrain = h['Mustrain']
2908                        mustrain[2] = [False for p in mustrain[2]]
2909                        mustrain[5] = [False for p in mustrain[4]]
2910                elif key == 'Pref.Ori.':
2911                    for h in histograms:
2912                        h['Pref.Ori.'][2] = False
2913                elif key == 'Show':
2914                    for h in histograms:
2915                        h['Show'] = False
2916                elif key == 'Size':
2917                    for h in histograms:
2918                        size = h['Size']
2919                        size[2] = [False for p in size[2]]
2920                        size[5] = [False for p in size[5]]
2921                elif key == 'Use':
2922                    for h in histograms:
2923                        h['Use'] = False
2924                elif key == 'Scale':
2925                    for h in histograms:
2926                        h['Scale'][1] = False
2927                else:
2928                    print(u'Unknown HAP key: '+key)
2929
2930
2931##########################
2932# Command Line Interface #
2933##########################
2934# Each of these takes an argparse.Namespace object as their argument,
2935# representing the parsed command-line arguments for the relevant subcommand.
2936# The argument specification for each is in the subcommands dictionary (see
2937# below)
2938
2939commandhelp={}
2940commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
2941def create(args):
2942    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
2943
2944  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2945                                  [-i IPARAMS [IPARAMS ...]]
2946                                  [-p PHASES [PHASES ...]]
2947                                  filename
2948                                 
2949positional arguments::
2950
2951  filename              the project file to create. should end in .gpx
2952
2953optional arguments::
2954
2955  -h, --help            show this help message and exit
2956  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2957                        list of datafiles to add as histograms
2958  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2959                        instrument parameter file, must be one for every
2960                        histogram
2961  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2962                        list of phases to add. phases are automatically
2963                        associated with all histograms given.
2964
2965    """
2966    proj = G2Project(gpxname=args.filename)
2967
2968    hist_objs = []
2969    if args.histograms:
2970        for h,i in zip(args.histograms,args.iparams):
2971            print("Adding histogram from",h,"with instparm ",i)
2972            hist_objs.append(proj.add_powder_histogram(h, i))
2973
2974    if args.phases: 
2975        for p in args.phases:
2976            print("Adding phase from",p)
2977            proj.add_phase(p, histograms=hist_objs)
2978        print('Linking phase(s) to histogram(s):')
2979        for h in hist_objs:
2980            print ('   '+h.name)
2981
2982    proj.save()
2983
2984commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
2985def add(args):
2986    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
2987
2988  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2989                               [-i IPARAMS [IPARAMS ...]]
2990                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
2991                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
2992                               filename
2993
2994
2995positional arguments::
2996
2997  filename              the project file to open. Should end in .gpx
2998
2999optional arguments::
3000
3001  -h, --help            show this help message and exit
3002  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
3003                        list of datafiles to add as histograms
3004  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
3005                        instrument parameter file, must be one for every
3006                        histogram
3007  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
3008                        format hint for histogram import. Applies to all
3009                        histograms
3010  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
3011                        list of phases to add. phases are automatically
3012                        associated with all histograms given.
3013  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
3014                        format hint for phase import. Applies to all phases.
3015                        Example: -pf CIF
3016  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
3017                        list of histgram indices to associate with added
3018                        phases. If not specified, phases are associated with
3019                        all previously loaded histograms. Example: -l 2 3 4
3020   
3021    """
3022    proj = G2Project(args.filename)
3023
3024    if args.histograms:
3025        for h,i in zip(args.histograms,args.iparams):
3026            print("Adding histogram from",h,"with instparm ",i)
3027            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
3028
3029    if args.phases: 
3030        if not args.histlist:
3031            histlist = proj.histograms()
3032        else:
3033            histlist = [proj.histogram(i) for i in args.histlist]
3034
3035        for p in args.phases:
3036            print("Adding phase from",p)
3037            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
3038           
3039        if not args.histlist:
3040            print('Linking phase(s) to all histogram(s)')
3041        else:
3042            print('Linking phase(s) to histogram(s):')
3043            for h in histlist:
3044                print ('   '+h.name)
3045
3046    proj.save()
3047
3048
3049commandhelp["dump"] = "Shows the contents of a GSAS-II project"
3050def dump(args):
3051    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
3052
3053       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
3054
3055positional arguments::
3056
3057  files
3058
3059optional arguments::
3060
3061  -h, --help        show this help message and exit
3062  -d, --histograms  list histograms in files, overrides --raw
3063  -p, --phases      list phases in files, overrides --raw
3064  -r, --raw         dump raw file contents, default
3065 
3066    """
3067    if not args.histograms and not args.phases:
3068        args.raw = True
3069    if args.raw:
3070        import IPython.lib.pretty as pretty
3071
3072    for fname in args.files:
3073        if args.raw:
3074            proj, nameList = LoadDictFromProjFile(fname)
3075            print("file:", fname)
3076            print("names:", nameList)
3077            for key, val in proj.items():
3078                print(key, ":")
3079                pretty.pprint(val)
3080        else:
3081            proj = G2Project(fname)
3082            if args.histograms:
3083                hists = proj.histograms()
3084                for h in hists:
3085                    print(fname, "hist", h.id, h.name)
3086            if args.phases:
3087                phase_list = proj.phases()
3088                for p in phase_list:
3089                    print(fname, "phase", p.id, p.name)
3090
3091
3092commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
3093def IPyBrowse(args):
3094    """Load a .gpx file and then open a IPython shell to browse it::
3095
3096  usage: GSASIIscriptable.py browse [-h] files [files ...]
3097
3098positional arguments::
3099
3100  files       list of files to browse
3101
3102optional arguments::
3103
3104  -h, --help  show this help message and exit
3105
3106    """
3107    for fname in args.files:
3108        proj, nameList = LoadDictFromProjFile(fname)
3109        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
3110        GSASIIpath.IPyBreak_base(msg)
3111        break
3112
3113
3114commandhelp["refine"] = '''
3115Conducts refinements on GSAS-II projects according to a list of refinement
3116steps in a JSON dict
3117'''
3118def refine(args):
3119    """Implements the refine command-line subcommand:
3120    conducts refinements on GSAS-II projects according to a JSON refinement dict::
3121
3122        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
3123
3124positional arguments::
3125
3126  gpxfile      the project file to refine
3127  refinements  json file of refinements to apply. if not present refines file
3128               as-is
3129
3130optional arguments::
3131
3132  -h, --help   show this help message and exit
3133 
3134    """
3135    proj = G2Project(args.gpxfile)
3136    if args.refinements is None:
3137        proj.refine()
3138    else:
3139        import json
3140        with open(args.refinements) as refs:
3141            refs = json.load(refs)
3142        if type(refs) is not dict:
3143            raise G2ScriptException("Error: JSON object must be a dict.")
3144        if "code" in refs:
3145            print("executing code:\n|  ",'\n|  '.join(refs['code']))
3146            exec('\n'.join(refs['code']))
3147        proj.do_refinements(refs['refinements'])
3148
3149
3150commandhelp["seqrefine"] = "Not implemented. Placeholder for eventual sequential refinement implementation"
3151def seqrefine(args):
3152    """Future implementation for the seqrefine command-line subcommand """
3153    raise NotImplementedError("seqrefine is not yet implemented")
3154
3155
3156commandhelp["export"] = "Export phase as CIF"
3157def export(args):
3158    """Implements the export command-line subcommand: Exports phase as CIF::
3159
3160      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
3161
3162positional arguments::
3163
3164  gpxfile     the project file from which to export
3165  phase       identifier of phase to export
3166  exportfile  the .cif file to export to
3167
3168optional arguments::
3169
3170  -h, --help  show this help message and exit
3171
3172    """
3173    proj = G2Project(args.gpxfile)
3174    phase = proj.phase(args.phase)
3175    phase.export_CIF(args.exportfile)
3176
3177
3178def _args_kwargs(*args, **kwargs):
3179    return args, kwargs
3180
3181# A dictionary of the name of each subcommand, and a tuple
3182# of its associated function and a list of its arguments
3183# The arguments are passed directly to the add_argument() method
3184# of an argparse.ArgumentParser
3185
3186subcommands = {"create":
3187               (create, [_args_kwargs('filename',
3188                                      help='the project file to create. should end in .gpx'),
3189
3190                         _args_kwargs('-d', '--histograms',
3191                                      nargs='+',
3192                                      help='list of datafiles to add as histograms'),
3193                                     
3194                         _args_kwargs('-i', '--iparams',
3195                                      nargs='+',
3196                                      help='instrument parameter file, must be one'
3197                                           ' for every histogram'
3198                                      ),
3199
3200                         _args_kwargs('-p', '--phases',
3201                                      nargs='+',
3202                                      help='list of phases to add. phases are '
3203                                           'automatically associated with all '
3204                                           'histograms given.')]),
3205               "add": (add, [_args_kwargs('filename',
3206                                      help='the project file to open. Should end in .gpx'),
3207
3208                         _args_kwargs('-d', '--histograms',
3209                                      nargs='+',
3210                                      help='list of datafiles to add as histograms'),
3211                                     
3212                         _args_kwargs('-i', '--iparams',
3213                                      nargs='+',
3214                                      help='instrument parameter file, must be one'
3215                                           ' for every histogram'
3216                                      ),
3217                                     
3218                         _args_kwargs('-hf', '--histogramformat',
3219                                      help='format hint for histogram import. Applies to all'
3220                                           ' histograms'
3221                                      ),
3222
3223                         _args_kwargs('-p', '--phases',
3224                                      nargs='+',
3225                                      help='list of phases to add. phases are '
3226                                           'automatically associated with all '
3227                                           'histograms given.'),
3228
3229                         _args_kwargs('-pf', '--phaseformat',
3230                                      help='format hint for phase import. Applies to all'
3231                                           ' phases. Example: -pf CIF'
3232                                      ),
3233                                     
3234                         _args_kwargs('-l', '--histlist',
3235                                      nargs='+',
3236                                      help='list of histgram indices to associate with added'
3237                                           ' phases. If not specified, phases are'
3238                                           ' associated with all previously loaded'
3239                                           ' histograms. Example: -l 2 3 4')]),
3240                                           
3241               "dump": (dump, [_args_kwargs('-d', '--histograms',
3242                                     action='store_true',
3243                                     help='list histograms in files, overrides --raw'),
3244
3245                               _args_kwargs('-p', '--phases',
3246                                            action='store_true',
3247                                            help='list phases in files, overrides --raw'),
3248
3249                               _args_kwargs('-r', '--raw',
3250                                      action='store_true', help='dump raw file contents, default'),
3251
3252                               _args_kwargs('files', nargs='+')]),
3253
3254               "refine":
3255               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
3256                         _args_kwargs('refinements',
3257                                      help='JSON file of refinements to apply. if not present'
3258                                           ' refines file as-is',
3259                                      default=None,
3260                                      nargs='?')]),
3261
3262               "seqrefine": (seqrefine, []),
3263               "export": (export, [_args_kwargs('gpxfile',
3264                                                help='the project file from which to export'),
3265                                   _args_kwargs('phase', help='identifier of phase to export'),
3266                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
3267               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
3268                                                   help='list of files to browse')])}
3269
3270
3271def main():
3272    '''The command-line interface for calling GSASIIscriptable as a shell command,
3273    where it is expected to be called as::
3274
3275       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
3276
3277    The following subcommands are defined:
3278
3279        * create, see :func:`create`
3280        * add, see :func:`add`
3281        * dump, see :func:`dump`
3282        * refine, see :func:`refine`
3283        * seqrefine, see :func:`seqrefine`
3284        * export, :func:`export`
3285        * browse, see :func:`IPyBrowse`
3286
3287    .. seealso::
3288        :func:`create`
3289        :func:`add`
3290        :func:`dump`
3291        :func:`refine`
3292        :func:`seqrefine`
3293        :func:`export`
3294        :func:`IPyBrowse`
3295    '''
3296    parser = argparse.ArgumentParser(description=
3297        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
3298        )
3299    subs = parser.add_subparsers()
3300
3301    # Create all of the specified subparsers
3302    for name, (func, args) in subcommands.items():
3303        new_parser = subs.add_parser(name,help=commandhelp.get(name),
3304                                     description='Command "'+name+'" '+commandhelp.get(name))
3305        for listargs, kwds in args:
3306            new_parser.add_argument(*listargs, **kwds)
3307        new_parser.set_defaults(func=func)
3308
3309    # Parse and trigger subcommand
3310    result = parser.parse_args()
3311    result.func(result)
3312
3313if __name__ == '__main__':
3314    #fname='/tmp/corundum-template.gpx'
3315    #prj = G2Project(fname)
3316    main()
Note: See TracBrowser for help on using the repository browser.