source: trunk/GSASIIscriptable.py @ 3439

Last change on this file since 3439 was 3439, checked in by toby, 3 years ago

more scripting peaks stuff

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 133.1 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2018-06-19 16:35:53 +0000 (Tue, 19 Jun 2018) $
5# $Author: toby $
6# $Revision: 3439 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3439 2018-06-19 16:35:53Z toby $
9########### SVN repository information ###################
10#
11# TODO: integrate 2d data based on a model
12#
13"""
14*GSASIIscriptable: Scripting Interface*
15=======================================
16
17Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
18This file specifies several wrapper classes around GSAS-II data representations.
19They all inherit from :class:`G2ObjectWrapper`. The chief class is :class:`G2Project`,
20which represents an entire GSAS-II project and provides several methods to access
21phases, powder histograms, and execute Rietveld refinements. These routines can be
22accessed in two ways:
23the :ref:`CommandlineInterface` provides access a number of features without writing
24Python scripts of from the :ref:`API`, which allows much more versatile access from
25within Python.
26
27
28=====================
29Overview
30=====================
31The most commonly used routines are:
32
33:class:`G2Project`
34   Always needed: used to create a new project (.gpx) file or read in an existing one.
35
36:meth:`G2Project.add_powder_histogram`
37   Used to read in powder diffraction data to a project file.
38
39:meth:`G2Project.add_simulated_powder_histogram`
40   Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
41
42:meth:`G2Project.save`
43   Writes the current project to disk.
44
45:meth:`G2Project.do_refinements`
46   This is passed a list of dictionaries, where each dict defines a refinement step.
47   Passing a list with a single empty dict initiates a refinement with the current
48   parameters and flags.
49   
50:meth:`G2Project.set_refinement`
51   This is passed a single dict which is used to set parameters and flags.
52   These actions can be performed also in :meth:`G2Project.do_refinements`.
53
54Refinement dicts
55   A refinement dict sets up a single refinement step
56   (as used in :meth:`G2Project.do_refinements` and described in
57   :ref:`Project_dicts`).
58   It specifies parameter & refinement flag changes, which are usually followed by a refinement and
59   optionally by calls to locally-defined Python functions. The keys used in these dicts are
60   defined in the :ref:`Refinement_recipe` table, below.
61
62There are several ways to set parameters project using different level objects, as is
63are described in sections below, but the simplest way to access parameters and flags
64in a project is to use the above routines.
65
66=====================
67Refinement parameters
68=====================
69The most complex part of scripting GSAS-II comes in setting up the input to control refinements, which is
70described here.
71
72.. _Project_dicts:
73
74-----------------------------
75Project-level Parameter Dict
76-----------------------------
77
78As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
79which can be accessed individually by the objects that encapsulate individual phases and histograms
80but it will be usually simplest to create a composite dictionary
81that is used at the project-level. A dict is created with keys
82"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
83(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
84determine parameter values and will determine which parameters will be refined.
85
86The specific keys and subkeys that can be used are defined in tables
87:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
88
89Note that optionally a list of histograms and/or phases can be supplied in the call to
90:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
91phases and histograms.
92
93As an example:
94
95.. code-block::  python
96
97    pardict = {'set': { 'Limits': [0.8, 12.0],
98                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
99                       'Background': {'type': 'chebyschev', 'refine': True}},
100              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
101    my_project.set_refinement(pardict)
102   
103.. _Refinement_recipe:
104   
105------------------------
106Refinement recipe
107------------------------
108Building on the :ref:`Project_dicts`,
109it is possible to specify a sequence of refinement actions as a list of
110these dicts and supplying this list
111as an argument to :meth:`G2Project.do_refinements`.
112
113As an example, this code performs the same actions as in the example in the section above:
114
115.. code-block::  python
116   
117    pardict = {'set': { 'Limits': [0.8, 12.0],
118                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
119                       'Background': {'type': 'chebyschev', 'refine': True}},
120              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
121    my_project.do_refinements([pardict])
122
123However, in addition to setting a number of parameters, this example will perform a refinement as well.
124If more than one dict is specified in the list, as is done in this example,
125two refinement steps will be performed:
126
127.. code-block::  python
128
129    my_project.do_refinements([pardict,pardict1])
130
131
132The keys defined in the following table
133may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that now keys ``histograms``
134and ``phases`` are used to limit actions to specific sets of parameters within the project.
135
136========== ============================================================================
137key         explanation
138========== ============================================================================
139set                    Specifies a dict with keys and subkeys as described in the
140                       :ref:`Refinement_parameters_fmt` section. Items listed here
141                       will be set to be refined.
142clear                  Specifies a dict, as above for set, except that parameters are
143                       cleared and thus will not be refined.
144once                   Specifies a dict as above for set, except that parameters are
145                       set for the next cycle of refinement and are cleared once the
146                       refinement step is completed.
147skip                   Normally, once parameters are processed with a set/clear/once
148                       action(s), a refinement is started. If skip is defined as True
149                       (or any other value) the refinement step is not performed.
150output                 If a file name is specified for output is will be used to save
151                       the current refinement.
152histograms             Should contain a list of histogram(s) to be used for the
153                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
154                       :ref:`HAP_parameters_table`. Note that this will be
155                       ignored for :ref:`Phase_parameters_table`. Histograms may be
156                       specified as a list of strings [('PWDR ...'),...], indices
157                       [0,1,2] or as list of objects [hist1, hist2].
158phases                 Should contain a list of phase(s) to be used for the
159                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
160                       :ref:`HAP_parameters_table`. Note that this will be
161                       ignored for :ref:`Histogram_parameters_table`.
162                       Phases may be specified as a list of strings
163                       [('Phase name'),...], indices [0,1,2] or as list of objects
164                       [phase0, phase2].
165call                   Specifies a function to call after a refinement is completed.
166                       The value supplied can be the object (typically a function)
167                       that will be called or a string that will evaluate (in the
168                       namespace inside :meth:`G2Project.iter_refinements` where
169                       ``self`` references the project.)
170                       Nothing is called if this is not specified.
171callargs               Provides a list of arguments that will be passed to the function
172                       in call (if any). If call is defined and callargs is not, the
173                       current <tt>G2Project</tt> is passed as a single argument.
174========== ============================================================================
175
176An example that performs a series of refinement steps follows:
177
178.. code-block::  python
179
180    reflist = [
181            {"set": { "Limits": { "low": 0.7 },
182                      "Background": { "no. coeffs": 3,
183                                      "refine": True }}},
184            {"set": { "LeBail": True,
185                      "Cell": True }},
186            {"set": { "Sample Parameters": ["DisplaceX"]}},
187            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
188            {"set": { "Mustrain": { "type": "uniaxial",
189                                    "refine": "equatorial",
190                                    "direction": [0, 0, 1]}}},
191            {"set": { "Mustrain": { "type": "uniaxial",
192                                    "refine": "axial"}}},
193            {"clear": { "LeBail": True},
194             "set": { "Atoms": { "Mn": "X" }}},
195            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
196    my_project.do_refinements(reflist)
197   
198
199In this example, a separate refinement step will be performed for each dict in the list (since
200"skip" is not included).
201Note that in the second from last refinement step, parameters are both set and cleared.
202   
203.. _Refinement_parameters_kinds:
204
205----------------------------
206Refinement parameter types
207----------------------------
208
209Note that parameters and refinement flags used in GSAS-II fall into three classes:
210
211    * **Histogram**: There will be a set of these for each dataset loaded into a
212      project file. The parameters available depend on the type of histogram
213      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
214      include the overall scale factor, background, instrument and sample parameters;
215      see the :ref:`Histogram_parameters_table` table for a list of the histogram
216      parameters where access has been provided.
217     
218    * **Phase**: There will be a set of these for each phase loaded into a
219      project file. While some parameters are found in all types of phases,
220      others are only found in certain types (modulated, magnetic, protein...).
221      Typical phase parameters include unit cell lengths and atomic positions; see the
222      :ref:`Phase_parameters_table` table for a list of the phase     
223      parameters where access has been provided.
224     
225    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
226      that is associated with each phase, so that if there are ``N`` phases and ``M``
227      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
228      histograms are not linked to all phases.) Typical HAP parameters include the
229      phase fractions, sample microstrain and crystallite size broadening terms,
230      hydrostatic strain perturbations of the unit cell and preferred orientation
231      values.
232      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
233      been provided.
234
235.. _Refinement_parameters_fmt:
236
237=================================
238Specifying Refinement Parameters
239=================================
240
241Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
242where the details of these dicts are organized depends on the
243type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
244of keys (as described below) for each of the three types of parameters.
245
246.. _Histogram_parameters_table:
247
248--------------------
249Histogram parameters
250--------------------
251
252This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
253and :func:`G2PwdrData.clear_refinements`. As an example,
254
255.. code-block::  python
256
257   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
258                         "Sample Parameters": ["Scale"],
259                         "Limits": [10000, 40000]})
260
261With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
262``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
263key is used to define specific histograms. As an example:
264
265.. code-block::  python
266
267  gsas_proj.do_refinements([
268      {'set': {
269          'Background': {'no.coeffs': 3, 'refine': True},
270          'Sample Parameters': ['Scale'],
271          'Limits': [10000, 40000]},
272      'histograms': [1,2]}
273                            ])
274
275Note that below in the Instrument Parameters section,
276related profile parameters (such as U and V) are grouped together but
277separated by commas to save space in the table.
278
279.. tabularcolumns:: |l|l|p{3.5in}|
280
281===================== ====================  =================================================
282key                   subkey                explanation
283===================== ====================  =================================================
284Limits                                      The range of 2-theta (degrees) or TOF (in
285                                            microsec) range of values to use. Can
286                                            be either a dictionary of 'low' and/or 'high',
287                                            or a list of 2 items [low, high]
288\                     low                   Sets the low limit
289\                     high                  Sets the high limit
290
291Sample Parameters                           Should be provided as a **list** of subkeys
292                                            to set or clear, e.g. ['DisplaceX', 'Scale']
293\                     Absorption
294\                     Contrast
295\                     DisplaceX             Sample displacement along the X direction
296\                     DisplaceY             Sample displacement along the Y direction
297\                     Scale                 Histogram Scale factor
298
299Background                                  Sample background. If value is a boolean,
300                                            the background's 'refine' parameter is set
301                                            to the given boolean. Usually should be a
302                                            dictionary with any of the following keys:
303\                     type                  The background model, e.g. 'chebyschev'
304\                     refine                The value of the refine flag, boolean
305\                     no. coeffs            Number of coefficients to use, integer
306\                     coeffs                List of floats, literal values for background
307\                     FixedPoints           List of (2-theta, intensity) values for fixed points
308\                     fit fixed points      If True, triggers a fit to the fixed points to
309                                            be calculated. It is calculated when this key is
310                                            detected, regardless of calls to refine.
311
312Instrument Parameters                       As in Sample Paramters, provide as a **list** of
313                                            subkeys to
314                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
315\                     U, V, W               Gaussian peak profile terms
316\                     X, Y, Z               Lorentzian peak profile terms
317\                     alpha, beta-0,        TOF profile terms
318                      beta-1, beta-q,
319\                     sig-0, sig-1,         TOF profile terms
320                      sig-2, sig-q
321\                     difA, difB, difC      TOF Calibration constants
322\                     Zero                  Zero shift
323\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
324\                     Polariz.              Polarization parameter
325\                     Lam                   Lambda, the incident wavelength
326===================== ====================  =================================================
327
328.. _Phase_parameters_table:
329
330----------------
331Phase parameters
332----------------
333
334This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
335and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
336these parameters should be placed inside a dict with a key
337``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
338key is used to define specific phase(s).
339
340
341.. tabularcolumns:: |l|p{4.5in}|
342
343======= ==========================================================
344key                   explanation
345======= ==========================================================
346Cell                  Whether or not to refine the unit cell.
347Atoms                 Dictionary of atoms and refinement flags.
348                      Each key should be an atom label, e.g.
349                      'O3', 'Mn5', and each value should be
350                      a string defining what values to refine.
351                      Values can be any combination of 'F'
352                      for fractional occupancy, 'X' for position,
353                      and 'U' for Debye-Waller factor
354LeBail                Enables LeBail intensity extraction.
355======= ==========================================================
356
357
358.. _HAP_parameters_table:
359
360
361Histogram-and-phase parameters
362------------------------------
363
364This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
365and :func:`G2Phase.clear_HAP_refinements`. When supplied to
366:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
367``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
368unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
369
370.. tabularcolumns:: |l|l|p{3.5in}|
371
372=============  ==========  ============================================================
373key             subkey                 explanation
374=============  ==========  ============================================================
375Babinet                                Should be a **list** of the following
376                                       subkeys. If not, assumes both
377                                       BabA and BabU
378\               BabA
379\               BabU
380Extinction                             Boolean, True to refine.
381HStrain                                Boolean, True to refine all appropriate
382                                       $D_ij$ terms.
383Mustrain
384\               type                   Mustrain model. One of 'isotropic',
385                                       'uniaxial', or 'generalized'. Should always
386                                       be specified.
387\              direction               For uniaxial only. A list of three
388                                       integers,
389                                       the [hkl] direction of the axis.
390\               refine                 Usually boolean, set to True to refine.
391                                       When in doubt, set it to true.
392                                       For uniaxial model, can specify list
393                                       of 'axial' or 'equatorial' or a single
394                                       boolean sets both axial and equatorial.
395Size                                   Not yet implemented
396\               type                   Size broadening model. One of 'isotropic',
397                                       'uniaxial', or 'ellipsoid'. Should always
398                                       be specified.
399\              direction               For uniaxial only. A list of three
400                                       integers,
401                                       the [hkl] direction of the axis.
402\               refine                 Boolean, True to refine.
403Pref.Ori.                              Boolean, True to refine
404Show                                   Boolean, True to refine
405Use                                    Boolean, True to refine
406Scale                                  Phase fraction; Boolean, True to refine
407=============  ==========  ============================================================
408
409------------------------
410Histogram/Phase objects
411------------------------
412Each phase and powder histogram in a :class:`G2Project` object has an associated
413object. Parameters within each individual object can be turned on and off by calling
414:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
415for histogram parameters;
416:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
417for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
418:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
419
420.. code-block::  python
421
422    params = { 'Limits': [0.8, 12.0],
423               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
424               'Background': {'type': 'chebyschev', 'refine': True}}
425    some_histogram.set_refinements(params)
426
427Likewise to turn refinement flags on, use code such as this:
428
429.. code-block::  python
430
431    params = { 'Instrument Parameters': ['U', 'V', 'W']}
432    some_histogram.set_refinements(params)
433
434and to turn these refinement flags, off use this (Note that the
435``.clear_refinements()`` methods will usually will turn off refinement even
436if a refinement parameter is set in the dict to True.):
437
438.. code-block::  python
439
440    params = { 'Instrument Parameters': ['U', 'V', 'W']}
441    some_histogram.clear_refinements(params)
442
443For phase parameters, use code such as this:
444   
445.. code-block::  python
446
447    params = { 'LeBail': True, 'Cell': True,
448               'Atoms': { 'Mn1': 'X',
449                          'O3': 'XU',
450                          'V4': 'FXU'}}
451    some_histogram.set_refinements(params)
452
453
454and here is an example for HAP parameters:
455
456.. code-block::  python
457
458    params = { 'Babinet': 'BabA',
459               'Extinction': True,
460               'Mustrain': { 'type': 'uniaxial',
461                             'direction': [0, 0, 1],
462                             'refine': True}}
463    some_phase.set_HAP_refinements(params)
464
465Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
466
467.. _CommandlineInterface:
468
469=======================================
470GSASIIscriptable Command-line Interface
471=======================================
472
473The routines described above are intended to be called from a Python script, but an
474alternate way to access some of the same functionality is to
475invoke the ``GSASIIscriptable.py`` script from
476the command line usually from within a shell script or batch file. This
477will usually be done with a command such as::
478
479       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
480
481    The following subcommands are defined:
482
483        * create, see :func:`create`
484        * add, see :func:`add`
485        * dump, see :func:`dump`
486        * refine, see :func:`refine`
487        * seqrefine, see :func:`seqrefine`
488        * export, :func:`export`
489        * browse, see :func:`IPyBrowse`
490       
491Run::
492
493   python GSASIIscriptable.py --help
494
495to show the available subcommands, and inspect each subcommand with
496`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
497
498.. _JsonFormat:
499
500-------------------------
501Parameters in JSON files
502-------------------------
503
504The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
505a JSON format file
506(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
507This dict may have two keys:
508
509refinements:
510  This defines the a set of refinement steps in a JSON representation of a
511  :ref:`Refinement_recipe` list.
512
513code:
514  This optionally defines Python code that will be executed after the project is loaded,
515  but before the refinement is started. This can be used to execute Python code to change
516  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
517  project object is accessed with variable ``proj``) or to define code that will be called
518  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
519   
520JSON website: `Introducing JSON <http://json.org/>`_.
521
522.. _API:
523
524============================================================
525GSASIIscriptable Application Layer (API)
526============================================================
527
528The large number of classes and modules in this module are described below.
529Most commonly a script will create a G2Project object using :class:`G2Project` and then
530perform actions such as
531adding a histogram (method :meth:`G2Project.add_powder_histogram`),
532adding a phase (method :meth:`G2Project.add_phase`),
533or setting parameters and performing a refinement
534(method :meth:`G2Project.do_refinements`).
535
536In some cases, it may be easier to use
537methods inside :class:`G2PwdrData` or :class:`G2Phase` or these objects
538may offer more options.
539
540---------------------------------------------------------------
541Complete Documentation: All classes and functions
542---------------------------------------------------------------
543"""
544from __future__ import division, print_function
545import argparse
546import os.path as ospath
547import datetime as dt
548import sys
549import platform
550if '2' in platform.python_version_tuple()[0]:
551    import cPickle
552    strtypes = (str,unicode)
553else:
554    import _pickle as cPickle
555    strtypes = (str,bytes)
556import imp
557import copy
558import os
559import random as ran
560
561import numpy.ma as ma
562import scipy.interpolate as si
563import numpy as np
564import scipy as sp
565
566import GSASIIpath
567GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
568import GSASIIobj as G2obj
569import GSASIIpwd as G2pwd
570import GSASIIstrMain as G2strMain
571import GSASIIspc as G2spc
572import GSASIIElem as G2elem
573
574
575# Delay imports to not slow down small scripts
576G2fil = None
577PwdrDataReaders = []
578PhaseReaders = []
579exportersByExtension = {}
580'''Specifies the list of extensions that are supported for Powder data export'''
581
582def LoadG2fil():
583    """Delay importing this module, it is slow"""
584    global G2fil
585    if G2fil is None:
586        import GSASIIfiles
587        G2fil = GSASIIfiles
588        global PwdrDataReaders
589        global PhaseReaders
590        PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
591        PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
592        AllExporters = G2fil.LoadExportRoutines(None)
593        global exportersByExtension
594        exportersByExtension = {}
595        for obj in AllExporters:
596            try:
597                obj.Writer
598            except AttributeError:
599                continue
600            for typ in obj.exporttype:
601                if typ not in exportersByExtension:
602                    exportersByExtension[typ] = {obj.extension:obj}
603                else:
604                    exportersByExtension[typ][obj.extension] = obj
605
606def LoadDictFromProjFile(ProjFile):
607    '''Read a GSAS-II project file and load items to dictionary
608   
609    :param str ProjFile: GSAS-II project (name.gpx) full file name
610    :returns: Project,nameList, where
611
612      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
613        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
614        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
615      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
616
617    Example for fap.gpx::
618
619      Project = {                 #NB:dict order is not tree order
620        u'Phases':{'data':None,'fap':{phase dict}},
621        u'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
622        u'Rigid bodies':{'data': {rigid body dict}},
623        u'Covariance':{'data':{covariance data dict}},
624        u'Controls':{'data':{controls data dict}},
625        u'Notebook':{'data':[notebook list]},
626        u'Restraints':{'data':{restraint data dict}},
627        u'Constraints':{'data':{constraint data dict}}]
628        }
629      nameList = [                #NB: reproduces tree order
630        [u'Notebook',],
631        [u'Controls',],
632        [u'Covariance',],
633        [u'Constraints',],
634        [u'Restraints',],
635        [u'Rigid bodies',],
636        [u'PWDR FAP.XRA Bank 1',
637             u'Comments',
638             u'Limits',
639             u'Background',
640             u'Instrument Parameters',
641             u'Sample Parameters',
642             u'Peak List',
643             u'Index Peak List',
644             u'Unit Cells List',
645             u'Reflection Lists'],
646        [u'Phases', u'fap']
647        ]
648    '''
649    # Let IOError be thrown if file does not exist
650    # if not ospath.exists(ProjFile):
651    #     print ('\n*** Error attempt to open project file that does not exist:\n   '+
652    #         str(ProjFile))
653    #     return
654    file = open(ProjFile,'rb')
655    # print('loading from file: {}'.format(ProjFile))
656    Project = {}
657    nameList = []
658    try:
659        while True:
660            try:
661                data = cPickle.load(file)
662            except EOFError:
663                break
664            datum = data[0]
665            Project[datum[0]] = {'data':datum[1]}
666            nameList.append([datum[0],])
667            for datus in data[1:]:
668                Project[datum[0]][datus[0]] = datus[1]
669                nameList[-1].append(datus[0])
670        file.close()
671        # print('project load successful')
672    except:
673        raise IOError("Error reading file "+str(ProjFile)+". This is not a GSAS-II .gpx file")
674    finally:
675        file.close()
676    return Project,nameList
677
678def SaveDictToProjFile(Project,nameList,ProjFile):
679    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
680
681    :param dict Project: representation of gpx file following the GSAS-II
682        tree structure as described for LoadDictFromProjFile
683    :param list nameList: names of main tree entries & subentries used to reconstruct project file
684    :param str ProjFile: full file name for output project.gpx file (including extension)
685    '''
686    file = open(ProjFile,'wb')
687    try:
688        for name in nameList:
689            data = []
690            item = Project[name[0]]
691            data.append([name[0],item['data']])
692            for item2 in name[1:]:
693                data.append([item2,item[item2]])
694            cPickle.dump(data,file,1)
695    finally:
696        file.close()
697
698# def ImportPowder(reader,filename):
699#     '''Use a reader to import a powder diffraction data file
700
701#     :param str reader: a scriptable reader
702#     :param str filename: full name of powder data file; can be "multi-Bank" data
703
704#     :returns: list rdlist: list of reader objects containing powder data, one for each
705#         "Bank" of data encountered in file. Items in reader object of interest are:
706
707#           * rd.comments: list of str: comments found on powder file
708#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
709#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
710#     '''
711#     rdfile,rdpath,descr = imp.find_module(reader)
712#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
713#     rd = rdclass.GSAS_ReaderClass()
714#     if not rd.scriptable:
715#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
716#         return None
717#     rdlist = []
718#     if rd.ContentsValidator(filename):
719#         repeat = True
720#         rdbuffer = {} # create temporary storage for file reader
721#         block = 0
722#         while repeat: # loop if the reader asks for another pass on the file
723#             block += 1
724#             repeat = False
725#             rd.objname = ospath.basename(filename)
726#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
727#             if flag:
728#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
729#                 if rd.repeat:
730#                     repeat = True
731#         return rdlist
732#     print(rd.errors)
733#     return None
734
735def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
736    '''Create an initial Histogram dictionary
737
738    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
739    '''
740    if dType in ['SXC','SNC']:
741        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
742            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
743            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
744            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
745            'Flack':[0.0,False]}
746    elif dType == 'SNT':
747        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
748            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
749            'Extinction':['Lorentzian','None', {
750            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
751    elif 'P' in dType:
752        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
753            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
754            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
755                [1.,1.,1.,0.,0.,0.],6*[False,]],
756            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
757                NShkl*[0.01,],NShkl*[False,]],
758            'HStrain':[NDij*[0.0,],NDij*[False,]],
759            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
760
761
762def PreSetup(data):
763    '''Create part of an initial (empty) phase dictionary
764
765    from GSASIIphsGUI.py, near end of UpdatePhaseData
766
767    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
768    '''
769    if 'RBModels' not in data:
770        data['RBModels'] = {}
771    if 'MCSA' not in data:
772        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
773    if 'dict' in str(type(data['MCSA']['Results'])):
774        data['MCSA']['Results'] = []
775    if 'Modulated' not in data['General']:
776        data['General']['Modulated'] = False
777#    if 'modulated' in data['General']['Type']:
778#        data['General']['Modulated'] = True
779#        data['General']['Type'] = 'nuclear'
780
781
782def SetupGeneral(data, dirname):
783    """Helps initialize phase data.
784
785    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
786
787    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
788    """
789    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
790                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
791    generalData = data['General']
792    atomData = data['Atoms']
793    generalData['AtomTypes'] = []
794    generalData['Isotopes'] = {}
795
796    if 'Isotope' not in generalData:
797        generalData['Isotope'] = {}
798    if 'Data plot type' not in generalData:
799        generalData['Data plot type'] = 'Mustrain'
800    if 'POhkl' not in generalData:
801        generalData['POhkl'] = [0,0,1]
802    if 'Map' not in generalData:
803        generalData['Map'] = mapDefault.copy()
804    if 'Flip' not in generalData:
805        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
806            'k-factor':0.1,'k-Max':20.,}
807    if 'testHKL' not in generalData['Flip']:
808        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
809    if 'doPawley' not in generalData:
810        generalData['doPawley'] = False     #ToDo: change to ''
811    if 'Pawley dmin' not in generalData:
812        generalData['Pawley dmin'] = 1.0
813    if 'Pawley neg wt' not in generalData:
814        generalData['Pawley neg wt'] = 0.0
815    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
816        'MCSA controls' not in generalData:
817        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
818        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
819        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
820    if 'AtomPtrs' not in generalData:
821        generalData['AtomPtrs'] = [3,1,7,9]
822        if generalData['Type'] == 'macromolecular':
823            generalData['AtomPtrs'] = [6,4,10,12]
824        elif generalData['Type'] == 'magnetic':
825            generalData['AtomPtrs'] = [3,1,10,12]
826    if generalData['Modulated']:
827        generalData['Type'] = 'nuclear'
828        if 'Super' not in generalData:
829            generalData['Super'] = 1
830            generalData['SuperVec'] = [[0,0,.1],False,4]
831            generalData['SSGData'] = {}
832        if '4DmapData' not in generalData:
833            generalData['4DmapData'] = mapDefault.copy()
834            generalData['4DmapData'].update({'MapType':'Fobs'})
835    if 'Modulated' not in generalData:
836        generalData['Modulated'] = False
837    if 'HydIds' not in generalData:
838        generalData['HydIds'] = {}
839    cx,ct,cs,cia = generalData['AtomPtrs']
840    generalData['NoAtoms'] = {}
841    generalData['BondRadii'] = []
842    generalData['AngleRadii'] = []
843    generalData['vdWRadii'] = []
844    generalData['AtomMass'] = []
845    generalData['Color'] = []
846    if generalData['Type'] == 'magnetic':
847        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
848        landeg = generalData.get('Lande g',[])
849    generalData['Mydir'] = dirname
850    badList = {}
851    for iat,atom in enumerate(atomData):
852        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
853        if generalData['AtomTypes'].count(atom[ct]):
854            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
855        elif atom[ct] != 'UNK':
856            Info = G2elem.GetAtomInfo(atom[ct])
857            if not Info:
858                if atom[ct] not in badList:
859                    badList[atom[ct]] = 0
860                badList[atom[ct]] += 1
861                atom[ct] = 'UNK'
862                continue
863            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
864            generalData['AtomTypes'].append(atom[ct])
865            generalData['Z'] = Info['Z']
866            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
867            generalData['BondRadii'].append(Info['Drad'])
868            generalData['AngleRadii'].append(Info['Arad'])
869            generalData['vdWRadii'].append(Info['Vdrad'])
870            if atom[ct] in generalData['Isotope']:
871                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
872                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
873                    generalData['Isotope'][atom[ct]] = isotope
874                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
875            else:
876                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
877                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
878                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
879                    generalData['Isotope'][atom[ct]] = isotope
880                generalData['AtomMass'].append(Info['Mass'])
881            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
882            generalData['Color'].append(Info['Color'])
883            if generalData['Type'] == 'magnetic':
884                if len(landeg) < len(generalData['AtomTypes']):
885                    landeg.append(2.0)
886    if generalData['Type'] == 'magnetic':
887        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
888
889    if badList:
890        msg = 'Warning: element symbol(s) not found:'
891        for key in badList:
892            msg += '\n\t' + key
893            if badList[key] > 1:
894                msg += ' (' + str(badList[key]) + ' times)'
895        raise G2ScriptException("Phase error:\n" + msg)
896        # wx.MessageBox(msg,caption='Element symbol error')
897    F000X = 0.
898    F000N = 0.
899    for i,elem in enumerate(generalData['AtomTypes']):
900        F000X += generalData['NoAtoms'][elem]*generalData['Z']
901        isotope = generalData['Isotope'][elem]
902        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
903    generalData['F000X'] = F000X
904    generalData['F000N'] = F000N
905    import GSASIImath as G2mth
906    generalData['Mass'] = G2mth.getMass(generalData)
907
908
909def make_empty_project(author=None, filename=None):
910    """Creates an dictionary in the style of GSASIIscriptable, for an empty
911    project.
912
913    If no author name or filename is supplied, 'no name' and
914    <current dir>/test_output.gpx are used , respectively.
915
916    Returns: project dictionary, name list
917
918    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
919    """
920    if not filename:
921        filename = 'test_output.gpx'
922    filename = os.path.abspath(filename)
923    gsasii_version = str(GSASIIpath.GetVersionNumber())
924    LoadG2fil()
925    import matplotlib as mpl
926    python_library_versions = G2fil.get_python_versions([mpl, np, sp])
927
928    controls_data = dict(G2obj.DefaultControls)
929    controls_data['LastSavedAs'] = filename
930    controls_data['LastSavedUsing'] = gsasii_version
931    controls_data['PythonVersions'] = python_library_versions
932    if author:
933        controls_data['Author'] = author
934
935    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
936                                       'Global': []}},
937              'Controls': {'data': controls_data},
938              u'Covariance': {'data': {}},
939              u'Notebook': {'data': ['']},
940              u'Restraints': {'data': {}},
941              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
942                                'Residue': {'AtInfo': {}},
943                                'Vector':  {'AtInfo': {}}}}}
944
945    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
946             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
947
948    return output, names
949
950
951class G2ImportException(Exception):
952    pass
953
954class G2ScriptException(Exception):
955    pass
956
957def import_generic(filename, readerlist, fmthint=None):
958    """Attempt to import a filename, using a list of reader objects.
959
960    Returns the first reader object which worked."""
961    # Translated from OnImportGeneric method in GSASII.py
962    primaryReaders, secondaryReaders = [], []
963    for reader in readerlist:
964        if fmthint is not None and fmthint not in reader.formatName: continue
965        flag = reader.ExtensionValidator(filename)
966        if flag is None:
967            secondaryReaders.append(reader)
968        elif flag:
969            primaryReaders.append(reader)
970    if not secondaryReaders and not primaryReaders:
971        raise G2ImportException("Could not read file: ", filename)
972
973    with open(filename, 'Ur') as fp:
974        rd_list = []
975
976        for rd in primaryReaders + secondaryReaders:
977            # Initialize reader
978            rd.selections = []
979            rd.dnames = []
980            rd.ReInitialize()
981            # Rewind file
982            rd.errors = ""
983            if not rd.ContentsValidator(filename):
984                # Report error
985                pass
986            if len(rd.selections) > 1:
987                # Select data?
988                # GSASII.py:543
989                raise G2ImportException("Not sure what data to select")
990
991            block = 0
992            rdbuffer = {}
993            repeat = True
994            while repeat:
995                repeat = False
996                block += 1
997                rd.objname = os.path.basename(filename)
998                try:
999                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
1000                except:
1001                    flag = False
1002                if flag:
1003                    # Omitting image loading special cases
1004                    rd.readfilename = filename
1005                    rd_list.append(copy.deepcopy(rd))
1006                    repeat = rd.repeat
1007                else:
1008                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
1009            if rd_list:
1010                if rd.warnings:
1011                    print("Read warning by", rd.formatName, "reader:",
1012                          rd.warnings, file=sys.stderr)
1013                else:
1014                    print("{} read by Reader {}\n".format(filename,rd.formatName))                   
1015                return rd_list
1016    raise G2ImportException("No reader could read file: " + filename)
1017
1018
1019def load_iprms(instfile, reader):
1020    """Loads instrument parameters from a file, and edits the
1021    given reader.
1022
1023    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1024    """
1025    LoadG2fil()
1026    ext = os.path.splitext(instfile)[1]
1027
1028    if ext.lower() == '.instprm':
1029        # New GSAS File, load appropriately
1030        with open(instfile) as f:
1031            lines = f.readlines()
1032        bank = reader.powderentry[2]
1033        numbanks = reader.numbanks
1034        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1035
1036        reader.instfile = instfile
1037        reader.instmsg = 'GSAS-II file' + instfile
1038        return iparms
1039    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1040        raise ValueError('Expected .prm file, found: ', instfile)
1041
1042    # It's an old GSAS file, load appropriately
1043    Iparm = {}
1044    with open(instfile, 'Ur') as fp:
1045        for line in fp:
1046            if '#' in line:
1047                continue
1048            Iparm[line[:12]] = line[12:-1]
1049    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1050    if ibanks == 1:
1051        reader.instbank = 1
1052        reader.powderentry[2] = 1
1053        reader.instfile = instfile
1054        reader.instmsg = instfile + ' bank ' + str(reader.instbank)
1055        return G2fil.SetPowderInstParms(Iparm, reader)
1056    # TODO handle >1 banks
1057    raise NotImplementedError("Check GSASIIfiles.py: ReadPowderInstprm")
1058
1059def load_pwd_from_reader(reader, instprm, existingnames=[]):
1060    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1061
1062    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1063
1064    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1065    """
1066    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1067    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1068
1069    try:
1070        Iparm1, Iparm2 = instprm
1071    except ValueError:
1072        Iparm1, Iparm2 = load_iprms(instprm, reader)
1073
1074    Ymin = np.min(reader.powderdata[1])
1075    Ymax = np.max(reader.powderdata[1])
1076    valuesdict = {'wtFactor': 1.0,
1077                  'Dummy': False,
1078                  'ranId': ran.randint(0, sys.maxsize),
1079                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1080                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1081                  'Yminmax': [Ymin, Ymax]}
1082    reader.Sample['ranId'] = valuesdict['ranId']
1083
1084    # Ending keys:
1085    # [u'Reflection Lists',
1086    #  u'Limits',
1087    #  'data',
1088    #  u'Index Peak List',
1089    #  u'Comments',
1090    #  u'Unit Cells List',
1091    #  u'Sample Parameters',
1092    #  u'Peak List',
1093    #  u'Background',
1094    #  u'Instrument Parameters']
1095    Tmin = np.min(reader.powderdata[0])
1096    Tmax = np.max(reader.powderdata[0])
1097
1098    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1099                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1100
1101    output_dict = {u'Reflection Lists': {},
1102                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1103                   u'data': [valuesdict, reader.powderdata, HistName],
1104                   u'Index Peak List': [[], []],
1105                   u'Comments': reader.comments,
1106                   u'Unit Cells List': [],
1107                   u'Sample Parameters': reader.Sample,
1108                   u'Peak List': {'peaks': [], 'sigDict': {}},
1109                   u'Background': reader.pwdparms.get('Background', default_background),
1110                   u'Instrument Parameters': [Iparm1, Iparm2],
1111                   }
1112
1113    names = [u'Comments',
1114             u'Limits',
1115             u'Background',
1116             u'Instrument Parameters',
1117             u'Sample Parameters',
1118             u'Peak List',
1119             u'Index Peak List',
1120             u'Unit Cells List',
1121             u'Reflection Lists']
1122
1123    # TODO controls?? GSASII.py:1664-7
1124
1125    return HistName, [HistName] + names, output_dict
1126
1127
1128def _deep_copy_into(from_, into):
1129    """Helper function for reloading .gpx file. See G2Project.reload()
1130
1131    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1132    """
1133    if isinstance(from_, dict) and isinstance(into, dict):
1134        combined_keys = set(from_.keys()).union(into.keys())
1135        for key in combined_keys:
1136            if key in from_ and key in into:
1137                both_dicts = (isinstance(from_[key], dict)
1138                              and isinstance(into[key], dict))
1139                both_lists = (isinstance(from_[key], list)
1140                              and isinstance(into[key], list))
1141                if both_dicts or both_lists:
1142                    _deep_copy_into(from_[key], into[key])
1143                else:
1144                    into[key] = from_[key]
1145            elif key in from_:
1146                into[key] = from_[key]
1147            else:  # key in into
1148                del into[key]
1149    elif isinstance(from_, list) and isinstance(into, list):
1150        if len(from_) == len(into):
1151            for i in range(len(from_)):
1152                both_dicts = (isinstance(from_[i], dict)
1153                              and isinstance(into[i], dict))
1154                both_lists = (isinstance(from_[i], list)
1155                              and isinstance(into[i], list))
1156                if both_dicts or both_lists:
1157                    _deep_copy_into(from_[i], into[i])
1158                else:
1159                    into[i] = from_[i]
1160        else:
1161            into[:] = from_
1162
1163
1164class G2ObjectWrapper(object):
1165    """Base class for all GSAS-II object wrappers.
1166
1167    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1168    of overrides are implemented so that the wrapper behaves like a dictionary.
1169
1170    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1171    """
1172    def __init__(self, datadict):
1173        self.data = datadict
1174
1175    def __getitem__(self, key):
1176        return self.data[key]
1177
1178    def __setitem__(self, key, value):
1179        self.data[key] = value
1180
1181    def __contains__(self, key):
1182        return key in self.data
1183
1184    def get(self, k, d=None):
1185        return self.data.get(k, d)
1186
1187    def keys(self):
1188        return self.data.keys()
1189
1190    def values(self):
1191        return self.data.values()
1192
1193    def items(self):
1194        return self.data.items()
1195
1196
1197class G2Project(G2ObjectWrapper):
1198    """
1199    Represents an entire GSAS-II project.
1200
1201    There are two ways to initialize it:
1202
1203    >>> # Load an existing project file
1204    >>> proj = G2Project('filename.gpx')
1205   
1206    >>> # Create a new project
1207    >>> proj = G2Project(newgpx='new_file.gpx')
1208   
1209    Histograms can be accessed easily.
1210
1211    >>> # By name
1212    >>> hist = proj.histogram('PWDR my-histogram-name')
1213   
1214    >>> # Or by index
1215    >>> hist = proj.histogram(0)
1216    >>> assert hist.id == 0
1217   
1218    >>> # Or by random id
1219    >>> assert hist == proj.histogram(hist.ranId)
1220
1221    Phases can be accessed the same way.
1222
1223    >>> phase = proj.phase('name of phase')
1224
1225    New data can also be loaded via :meth:`~G2Project.add_phase` and
1226    :meth:`~G2Project.add_powder_histogram`.
1227
1228    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1229                                         'instrument_parameters.prm')
1230    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1231
1232    Parameters for Rietveld refinement can be turned on and off as well.
1233    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1234    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1235    """
1236    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1237        """Loads a GSAS-II project from a specified filename.
1238
1239        :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1240            creates an empty project.
1241        :param str author: Author's name (not yet implemented)
1242        :param str newgpx: The filename the project should be saved to in
1243            the future. If both newgpx and gpxfile are present, the project is
1244            loaded from the gpxfile, then when saved will be written to newgpx.
1245        :param str filename: Name to be used to save the project. Has same function as
1246            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1247            is preferred over filename.
1248        """
1249        if filename is not None and newgpx is not None:
1250            raise G2ScriptException('Do not use filename and newgpx together')
1251        elif newgpx is not None:
1252            filename = newgpx
1253        if gpxfile is None:
1254            filename = os.path.abspath(os.path.expanduser(filename))
1255            self.filename = filename
1256            self.data, self.names = make_empty_project(author=author, filename=filename)
1257        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1258                                       # filename is valid, etc.
1259            if isinstance(filename, str): 
1260                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1261            else: 
1262                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1263            # TODO set author
1264            self.data, self.names = LoadDictFromProjFile(gpxfile)
1265            self.update_ids()
1266        else:
1267            raise ValueError("Not sure what to do with gpxfile")
1268
1269    @classmethod
1270    def from_dict_and_names(cls, gpxdict, names, filename=None):
1271        """Creates a :class:`G2Project` directly from
1272        a dictionary and a list of names. If in doubt, do not use this.
1273
1274        :returns: a :class:`G2Project`
1275        """
1276        out = cls()
1277        if filename:
1278            filename = os.path.abspath(os.path.expanduser(filename))
1279            out.filename = filename
1280            gpxdict['Controls']['data']['LastSavedAs'] = filename
1281        else:
1282            try:
1283                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1284            except KeyError:
1285                out.filename = None
1286        out.data = gpxdict
1287        out.names = names
1288
1289    def save(self, filename=None):
1290        """Saves the project, either to the current filename, or to a new file.
1291
1292        Updates self.filename if a new filename provided"""
1293        # TODO update LastSavedUsing ?
1294        if filename:
1295            filename = os.path.abspath(os.path.expanduser(filename))
1296            self.data['Controls']['data']['LastSavedAs'] = filename
1297            self.filename = filename
1298        elif not self.filename:
1299            raise AttributeError("No file name to save to")
1300        SaveDictToProjFile(self.data, self.names, self.filename)
1301
1302    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None):
1303        """Loads a powder data histogram into the project.
1304
1305        Automatically checks for an instrument parameter file, or one can be
1306        provided. Note that in unix fashion, "~" can be used to indicate the
1307        home directory (e.g. ~/G2data/data.fxye).
1308
1309        :param str datafile: The powder data file to read, a filename.
1310        :param str iparams: The instrument parameters file, a filename.
1311        :param list phases: Phases to link to the new histogram
1312        :param str fmthint: If specified, only importers where the format name
1313          (reader.formatName, as shown in Import menu) contains the
1314          supplied string will be tried as importers. If not specified, all
1315          importers consistent with the file extension will be tried
1316          (equivalent to "guess format" in menu).
1317
1318        :returns: A :class:`G2PwdrData` object representing
1319            the histogram
1320        """
1321        LoadG2fil()
1322        datafile = os.path.abspath(os.path.expanduser(datafile))
1323        iparams = os.path.abspath(os.path.expanduser(iparams))
1324        pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint)
1325        histname, new_names, pwdrdata = load_pwd_from_reader(
1326                                          pwdrreaders[0], iparams,
1327                                          [h.name for h in self.histograms()])
1328        if histname in self.data:
1329            print("Warning - redefining histogram", histname)
1330        elif self.names[-1][0] == 'Phases':
1331            self.names.insert(-1, new_names)
1332        else:
1333            self.names.append(new_names)
1334        self.data[histname] = pwdrdata
1335        self.update_ids()
1336
1337        for phase in phases:
1338            phase = self.phase(phase)
1339            self.link_histogram_phase(histname, phase)
1340
1341        return self.histogram(histname)
1342
1343    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1344                                       wavelength=None, scale=None, phases=[]):
1345        """Loads a powder data histogram into the project.
1346
1347        Requires an instrument parameter file.
1348        Note that in unix fashion, "~" can be used to indicate the
1349        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1350        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1351        as the instrument type.
1352
1353        :param str histname: A name for the histogram to be created.
1354        :param str iparams: The instrument parameters file, a filename.
1355        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1356        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1357        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1358        :param float wavelength: Wavelength for CW instruments, overriding the value
1359           in the instrument parameters file if specified.
1360        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1361           simulated noise is added to the pattern, so that if the maximum intensity is
1362           small, the noise will mask the computed pattern. The scale
1363           needs to be a large number for CW neutrons.
1364           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1365        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1366           all defined phases.
1367
1368        :returns: A :class:`G2PwdrData` object representing the histogram
1369        """
1370        LoadG2fil()
1371        iparams = os.path.abspath(os.path.expanduser(iparams))
1372        if not os.path.exists(iparams):
1373            raise G2ScriptException("File does not exist:"+iparams)
1374        rd = G2obj.ImportPowderData( # Initialize a base class reader
1375            extensionlist=tuple(),
1376            strictExtension=False,
1377            formatName = 'Simulate dataset',
1378            longFormatName = 'Compute a simulated pattern')
1379        rd.powderentry[0] = '' # no filename
1380        rd.powderentry[2] = 1 # only one bank
1381        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1382        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1383        if Tmax < Tmin:
1384            Tmin,Tmax = Tmax,Tmin
1385        Tstep = abs(Tstep)
1386        if 'TOF' in rd.idstring:
1387                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1388                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1389                N = len(x)
1390        else:           
1391                N = int((Tmax-Tmin)/Tstep)+1
1392                x = np.linspace(Tmin,Tmax,N,True)
1393                N = len(x)
1394        if N < 3:
1395            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1396        rd.powderdata = [
1397            np.array(x), # x-axis values
1398            np.zeros_like(x), # powder pattern intensities
1399            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1400            np.zeros_like(x), # calc. intensities (zero)
1401            np.zeros_like(x), # calc. background (zero)
1402            np.zeros_like(x), # obs-calc profiles
1403            ]
1404        Tmin = rd.powderdata[0][0]
1405        Tmax = rd.powderdata[0][-1]
1406        rd.idstring = histname
1407        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1408                                                            [h.name for h in self.histograms()])
1409        if histname in self.data:
1410            print("Warning - redefining histogram", histname)
1411        elif self.names[-1][0] == 'Phases':
1412            self.names.insert(-1, new_names)
1413        else:
1414            self.names.append(new_names)
1415        if scale is not None:
1416            pwdrdata['Sample Parameters']['Scale'][0] = scale
1417        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1418            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1419        self.data[histname] = pwdrdata
1420        self.update_ids()
1421
1422        for phase in phases:
1423            phase = self.phase(phase)
1424            self.link_histogram_phase(histname, phase)
1425
1426        return self.histogram(histname)
1427   
1428    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1429        """Loads a phase into the project from a .cif file
1430
1431        :param str phasefile: The CIF file from which to import the phase.
1432        :param str phasename: The name of the new phase, or None for the default
1433        :param list histograms: The names of the histograms to associate with
1434            this phase. Use proj.Histograms() to add to all histograms.
1435        :param str fmthint: If specified, only importers where the format name
1436          (reader.formatName, as shown in Import menu) contains the
1437          supplied string will be tried as importers. If not specified, all
1438          importers consistent with the file extension will be tried
1439          (equivalent to "guess format" in menu).
1440
1441        :returns: A :class:`G2Phase` object representing the
1442            new phase.
1443        """
1444        LoadG2fil()
1445        histograms = [self.histogram(h).name for h in histograms]
1446        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1447
1448        # TODO handle multiple phases in a file
1449        phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
1450        phasereader = phasereaders[0]
1451       
1452        phasename = phasename or phasereader.Phase['General']['Name']
1453        phaseNameList = [p.name for p in self.phases()]
1454        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1455        phasereader.Phase['General']['Name'] = phasename
1456
1457        if 'Phases' not in self.data:
1458            self.data[u'Phases'] = { 'data': None }
1459        assert phasename not in self.data['Phases'], "phase names should be unique"
1460        self.data['Phases'][phasename] = phasereader.Phase
1461
1462        if phasereader.Constraints:
1463            Constraints = self.data['Constraints']
1464            for i in phasereader.Constraints:
1465                if isinstance(i, dict):
1466                    if '_Explain' not in Constraints:
1467                        Constraints['_Explain'] = {}
1468                    Constraints['_Explain'].update(i)
1469                else:
1470                    Constraints['Phase'].append(i)
1471
1472        data = self.data['Phases'][phasename]
1473        generalData = data['General']
1474        SGData = generalData['SGData']
1475        NShkl = len(G2spc.MustrainNames(SGData))
1476        NDij = len(G2spc.HStrainNames(SGData))
1477        Super = generalData.get('Super', 0)
1478        if Super:
1479            SuperVec = np.array(generalData['SuperVec'][0])
1480        else:
1481            SuperVec = []
1482        UseList = data['Histograms']
1483
1484        for hist in histograms:
1485            self.link_histogram_phase(hist, phasename)
1486
1487        for obj in self.names:
1488            if obj[0] == 'Phases':
1489                phasenames = obj
1490                break
1491        else:
1492            phasenames = [u'Phases']
1493            self.names.append(phasenames)
1494        phasenames.append(phasename)
1495
1496        # TODO should it be self.filename, not phasefile?
1497        SetupGeneral(data, os.path.dirname(phasefile))
1498        self.index_ids()
1499
1500        self.update_ids()
1501        return self.phase(phasename)
1502
1503    def link_histogram_phase(self, histogram, phase):
1504        """Associates a given histogram and phase.
1505
1506        .. seealso::
1507
1508            :meth:`G2Project.histogram`
1509            :meth:`G2Project.phase`"""
1510        hist = self.histogram(histogram)
1511        phase = self.phase(phase)
1512
1513        generalData = phase['General']
1514
1515        if hist.name.startswith('HKLF '):
1516            raise NotImplementedError("HKLF not yet supported")
1517        elif hist.name.startswith('PWDR '):
1518            hist['Reflection Lists'][generalData['Name']] = {}
1519            UseList = phase['Histograms']
1520            SGData = generalData['SGData']
1521            NShkl = len(G2spc.MustrainNames(SGData))
1522            NDij = len(G2spc.HStrainNames(SGData))
1523            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1524            UseList[hist.name]['hId'] = hist.id
1525            for key, val in [('Use', True), ('LeBail', False),
1526                             ('newLeBail', True),
1527                             ('Babinet', {'BabA': [0.0, False],
1528                                          'BabU': [0.0, False]})]:
1529                if key not in UseList[hist.name]:
1530                    UseList[hist.name][key] = val
1531        else:
1532            raise RuntimeError("Unexpected histogram" + hist.name)
1533
1534
1535    def reload(self):
1536        """Reload self from self.filename"""
1537        data, names = LoadDictFromProjFile(self.filename)
1538        self.names = names
1539        # Need to deep copy the new data file data into the current tree,
1540        # so that any existing G2Phase, or G2PwdrData objects will still be
1541        # valid
1542        _deep_copy_into(from_=data, into=self.data)
1543
1544    def refine(self, newfile=None, printFile=None, makeBack=False):
1545        # TODO migrate to RefineCore
1546        # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1547        #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1548        # index_ids will automatically save the project
1549        self.index_ids()
1550        # TODO G2strMain does not properly use printFile
1551        G2strMain.Refine(self.filename, makeBack=makeBack)
1552        # Reload yourself
1553        self.reload()
1554
1555    def histogram(self, histname):
1556        """Returns the histogram named histname, or None if it does not exist.
1557
1558        :param histname: The name of the histogram (str), or ranId or index.
1559        :returns: A :class:`G2PwdrData` object, or None if
1560            the histogram does not exist
1561
1562        .. seealso::
1563            :meth:`G2Project.histograms`
1564            :meth:`G2Project.phase`
1565            :meth:`G2Project.phases`
1566            """
1567        if isinstance(histname, G2PwdrData):
1568            if histname.proj == self:
1569                return histname
1570        if histname in self.data:
1571            return G2PwdrData(self.data[histname], self)
1572        try:
1573            # see if histname is an id or ranId
1574            histname = int(histname)
1575        except ValueError:
1576            return
1577
1578        for histogram in self.histograms():
1579            if histogram.id == histname or histogram.ranId == histname:
1580                return histogram
1581
1582    def histograms(self):
1583        """Return a list of all histograms, as
1584        :class:`G2PwdrData` objects
1585
1586        .. seealso::
1587            :meth:`G2Project.histograms`
1588            :meth:`G2Project.phase`
1589            :meth:`G2Project.phases`
1590            """
1591        output = []
1592        for obj in self.names:
1593            if len(obj) > 1 and obj[0] != u'Phases':
1594                output.append(self.histogram(obj[0]))
1595        return output
1596
1597    def phase(self, phasename):
1598        """
1599        Gives an object representing the specified phase in this project.
1600
1601        :param str phasename: A reference to the desired phase. Either the phase
1602            name (str), the phase's ranId, the phase's index (both int) or
1603            a phase object (:class:`G2Phase`)
1604        :returns: A :class:`G2Phase` object
1605        :raises: KeyError
1606
1607        .. seealso::
1608            :meth:`G2Project.histograms`
1609            :meth:`G2Project.phase`
1610            :meth:`G2Project.phases`
1611            """
1612        if isinstance(phasename, G2Phase):
1613            if phasename.proj == self:
1614                return phasename
1615        phases = self.data['Phases']
1616        if phasename in phases:
1617            return G2Phase(phases[phasename], phasename, self)
1618
1619        try:
1620            # phasename should be phase index or ranId
1621            phasename = int(phasename)
1622        except ValueError:
1623            return
1624
1625        for phase in self.phases():
1626            if phase.id == phasename or phase.ranId == phasename:
1627                return phase
1628
1629    def phases(self):
1630        """
1631        Returns a list of all the phases in the project.
1632
1633        :returns: A list of :class:`G2Phase` objects
1634
1635        .. seealso::
1636            :meth:`G2Project.histogram`
1637            :meth:`G2Project.histograms`
1638            :meth:`G2Project.phase`
1639            """
1640        for obj in self.names:
1641            if obj[0] == 'Phases':
1642                return [self.phase(p) for p in obj[1:]]
1643        return []
1644
1645    def update_ids(self):
1646        """Makes sure all phases and histograms have proper hId and pId"""
1647        # Translated from GetUsedHistogramsAndPhasesfromTree,
1648        #   GSASIIdataGUI.py:4107
1649        for i, h in enumerate(self.histograms()):
1650            h.id = i
1651        for i, p in enumerate(self.phases()):
1652            p.id = i
1653
1654    def do_refinements(self, refinements, histogram='all', phase='all',
1655                       outputnames=None, makeBack=False):
1656        """Conducts one or a series of refinements according to the
1657           input provided in parameter refinements. This is a wrapper
1658           around :meth:`iter_refinements`
1659
1660        :param list refinements: A list of dictionaries specifiying changes to be made to
1661            parameters before refinements are conducted.
1662            See the :ref:`Refinement_recipe` section for how this is defined.
1663        :param str histogram: Name of histogram for refinements to be applied
1664            to, or 'all'; note that this can be overridden for each refinement
1665            step via a "histograms" entry in the dict.
1666        :param str phase: Name of phase for refinements to be applied to, or
1667            'all'; note that this can be overridden for each refinement
1668            step via a "phases" entry in the dict.
1669        :param list outputnames: Provides a list of project (.gpx) file names
1670            to use for each refinement step (specifying None skips the save step).
1671            See :meth:`save`.
1672            Note that this can be overridden using an "output" entry in the dict.
1673        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
1674            before a refinement is performed. The default is False.
1675           
1676        To perform a single refinement without changing any parameters, use this
1677        call:
1678
1679        .. code-block::  python
1680       
1681            my_project.do_refinements([])
1682        """
1683       
1684        for proj in self.iter_refinements(refinements, histogram, phase,
1685                                          outputnames, makeBack):
1686            pass
1687        return self
1688
1689    def iter_refinements(self, refinements, histogram='all', phase='all',
1690                         outputnames=None, makeBack=False):
1691        """Conducts a series of refinements, iteratively. Stops after every
1692        refinement and yields this project, to allow error checking or
1693        logging of intermediate results. Parameter use is the same as for
1694        :meth:`do_refinements` (which calls this method).
1695
1696        >>> def checked_refinements(proj):
1697        ...     for p in proj.iter_refinements(refs):
1698        ...         # Track intermediate results
1699        ...         log(p.histogram('0').residuals)
1700        ...         log(p.phase('0').get_cell())
1701        ...         # Check if parameter diverged, nonsense answer, or whatever
1702        ...         if is_something_wrong(p):
1703        ...             raise Exception("I need a human!")
1704
1705           
1706        """
1707        if outputnames:
1708            if len(refinements) != len(outputnames):
1709                raise ValueError("Should have same number of outputs to"
1710                                 "refinements")
1711        else:
1712            outputnames = [None for r in refinements]
1713
1714        for output, refinedict in zip(outputnames, refinements):
1715            if 'histograms' in refinedict:
1716                hist = refinedict['histograms']
1717            else:
1718                hist = histogram
1719            if 'phases' in refinedict:
1720                ph = refinedict['phases']
1721            else:
1722                ph = phase
1723            if 'output' in refinedict:
1724                output = refinedict['output']
1725            self.set_refinement(refinedict, hist, ph)
1726            # Handle 'once' args - refinements that are disabled after this
1727            # refinement
1728            if 'once' in refinedict:
1729                temp = {'set': refinedict['once']}
1730                self.set_refinement(temp, hist, ph)
1731
1732            if output:
1733                self.save(output)
1734
1735            if 'skip' not in refinedict:
1736                self.refine(makeBack=makeBack)
1737            yield self
1738
1739            # Handle 'once' args - refinements that are disabled after this
1740            # refinement
1741            if 'once' in refinedict:
1742                temp = {'clear': refinedict['once']}
1743                self.set_refinement(temp, hist, ph)
1744            if 'call' in refinedict:
1745                fxn = refinedict['call']
1746                if callable(fxn):
1747                    fxn(*refinedict.get('callargs',[self]))
1748                elif callable(eval(fxn)):
1749                    eval(fxn)(*refinedict.get('callargs',[self]))
1750                else:
1751                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
1752
1753    def set_refinement(self, refinement, histogram='all', phase='all'):
1754        """Apply specified refinements to a given histogram(s) or phase(s).
1755
1756        :param dict refinement: The refinements to be conducted
1757        :param histogram: Specifies either 'all' (default), a single histogram or
1758          a list of histograms. Histograms may be specified as histogram objects
1759          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
1760          of the histogram in the project, numbered starting from 0.
1761          Omitting the parameter or the string 'all' indicates that parameters in
1762          all histograms should be set.
1763        :param phase: Specifies either 'all' (default), a single phase or
1764          a list of phases. Phases may be specified as phase objects
1765          (see :class:`G2Phase`), the phase name (str) or the index number (int)
1766          of the phase in the project, numbered starting from 0.
1767          Omitting the parameter or the string 'all' indicates that parameters in
1768          all phases should be set.
1769
1770        Note that refinement parameters are categorized as one of three types:
1771
1772        1. Histogram parameters
1773        2. Phase parameters
1774        3. Histogram-and-Phase (HAP) parameters
1775       
1776        .. seealso::
1777            :meth:`G2PwdrData.set_refinements`
1778            :meth:`G2PwdrData.clear_refinements`
1779            :meth:`G2Phase.set_refinements`
1780            :meth:`G2Phase.clear_refinements`
1781            :meth:`G2Phase.set_HAP_refinements`
1782            :meth:`G2Phase.clear_HAP_refinements`"""
1783
1784        if histogram == 'all':
1785            hists = self.histograms()
1786        elif isinstance(histogram, list) or isinstance(histogram, tuple):
1787            hists = []
1788            for h in histogram:
1789                if isinstance(h, str) or isinstance(h, int):
1790                    hists.append(self.histogram(h))
1791                else:
1792                    hists.append(h)
1793        elif isinstance(histogram, str) or isinstance(histogram, int):
1794            hists = [self.histogram(histogram)]
1795        else:
1796            hists = [histogram]
1797
1798        if phase == 'all':
1799            phases = self.phases()
1800        elif isinstance(phase, list) or isinstance(phase, tuple):
1801            phases = []
1802            for ph in phase:
1803                if isinstance(ph, str) or isinstance(ph, int):
1804                    phases.append(self.phase(ph))
1805                else:
1806                    phases.append(ph)
1807        elif isinstance(phase, str) or isinstance(phase, int):
1808            phases = [self.phase(phase)]
1809        else:
1810            phases = [phase]
1811
1812        # TODO: HAP parameters:
1813        #   Babinet
1814        #   Extinction
1815        #   HStrain
1816        #   Mustrain
1817        #   Pref. Ori
1818        #   Size
1819
1820        pwdr_set = {}
1821        phase_set = {}
1822        hap_set = {}
1823        for key, val in refinement.get('set', {}).items():
1824            # Apply refinement options
1825            if G2PwdrData.is_valid_refinement_key(key):
1826                pwdr_set[key] = val
1827            elif G2Phase.is_valid_refinement_key(key):
1828                phase_set[key] = val
1829            elif G2Phase.is_valid_HAP_refinement_key(key):
1830                hap_set[key] = val
1831            else:
1832                raise ValueError("Unknown refinement key", key)
1833
1834        for hist in hists:
1835            hist.set_refinements(pwdr_set)
1836        for phase in phases:
1837            phase.set_refinements(phase_set)
1838        for phase in phases:
1839            phase.set_HAP_refinements(hap_set, hists)
1840
1841        pwdr_clear = {}
1842        phase_clear = {}
1843        hap_clear = {}
1844        for key, val in refinement.get('clear', {}).items():
1845            # Clear refinement options
1846            if G2PwdrData.is_valid_refinement_key(key):
1847                pwdr_clear[key] = val
1848            elif G2Phase.is_valid_refinement_key(key):
1849                phase_clear[key] = val
1850            elif G2Phase.is_valid_HAP_refinement_key(key):
1851                hap_set[key] = val
1852            else:
1853                raise ValueError("Unknown refinement key", key)
1854
1855        for hist in hists:
1856            hist.clear_refinements(pwdr_clear)
1857        for phase in phases:
1858            phase.clear_refinements(phase_clear)
1859        for phase in phases:
1860            phase.clear_HAP_refinements(hap_clear, hists)
1861
1862    def index_ids(self):
1863        import GSASIIstrIO as G2strIO
1864        self.save()
1865        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
1866
1867    def add_constraint_raw(self, cons_scope, constr):
1868        """Adds a constraint of type consType to the project.
1869        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
1870
1871        WARNING it does not check the constraint is well-constructed"""
1872        constrs = self.data['Constraints']['data']
1873        if 'Global' not in constrs:
1874            constrs['Global'] = []
1875        constrs[cons_scope].append(constr)
1876
1877    def hold_many(self, vars, type):
1878        """Apply holds for all the variables in vars, for constraint of a given type.
1879
1880        type is passed directly to add_constraint_raw as consType
1881
1882        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
1883            string variable specifiers, or arguments for :meth:`make_var_obj`
1884        :param str type: A string constraint type specifier. See
1885            :class:`G2Project.add_constraint_raw`
1886
1887        """
1888        for var in vars:
1889            if isinstance(var, str):
1890                var = self.make_var_obj(var)
1891            elif not isinstance(var, G2obj.G2VarObj):
1892                var = self.make_var_obj(*var)
1893            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
1894
1895    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
1896                     reloadIdx=True):
1897        """Wrapper to create a G2VarObj. Takes either a string representaiton ("p:h:name:a")
1898        or individual names of phase, histogram, varname, and atomId.
1899
1900        Automatically converts string phase, hist, or atom names into the ID required
1901        by G2VarObj."""
1902
1903        if reloadIdx:
1904            self.index_ids()
1905
1906        # If string representation, short circuit
1907        if hist is None and varname is None and atomId is None:
1908            if isinstance(phase, str) and ':' in phase:
1909                return G2obj.G2VarObj(phase)
1910
1911        # Get phase index
1912        phaseObj = None
1913        if isinstance(phase, G2Phase):
1914            phaseObj = phase
1915            phase = G2obj.PhaseRanIdLookup[phase.ranId]
1916        elif phase in self.data['Phases']:
1917            phaseObj = self.phase(phase)
1918            phaseRanId = phaseObj.ranId
1919            phase = G2obj.PhaseRanIdLookup[phaseRanId]
1920
1921        # Get histogram index
1922        if isinstance(hist, G2PwdrData):
1923            hist = G2obj.HistRanIdLookup[hist.ranId]
1924        elif hist in self.data:
1925            histRanId = self.histogram(hist).ranId
1926            hist = G2obj.HistRanIdLookup[histRanId]
1927
1928        # Get atom index (if any)
1929        if isinstance(atomId, G2AtomRecord):
1930            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
1931        elif phaseObj:
1932            atomObj = phaseObj.atom(atomId)
1933            if atomObj:
1934                atomRanId = atomObj.ranId
1935                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
1936
1937        return G2obj.G2VarObj(phase, hist, varname, atomId)
1938
1939
1940class G2AtomRecord(G2ObjectWrapper):
1941    """Wrapper for an atom record. Has convenient accessors via @property.
1942
1943
1944    Available accessors: label, type, refinement_flags, coordinates,
1945        occupancy, ranId, id, adp_flag, uiso
1946
1947    Example:
1948
1949    >>> atom = some_phase.atom("O3")
1950    >>> # We can access the underlying data format
1951    >>> atom.data
1952    ['O3', 'O-2', '', ... ]
1953    >>> # We can also use wrapper accessors
1954    >>> atom.coordinates
1955    (0.33, 0.15, 0.5)
1956    >>> atom.refinement_flags
1957    u'FX'
1958    >>> atom.ranId
1959    4615973324315876477
1960    >>> atom.occupancy
1961    1.0
1962    """
1963    def __init__(self, data, indices, proj):
1964        self.data = data
1965        self.cx, self.ct, self.cs, self.cia = indices
1966        self.proj = proj
1967
1968    @property
1969    def label(self):
1970        return self.data[self.ct-1]
1971
1972    @property
1973    def type(self):
1974        return self.data[self.ct]
1975
1976    @property
1977    def refinement_flags(self):
1978        return self.data[self.ct+1]
1979
1980    @refinement_flags.setter
1981    def refinement_flags(self, other):
1982        # Automatically check it is a valid refinement
1983        for c in other:
1984            if c not in ' FXU':
1985                raise ValueError("Invalid atom refinement: ", other)
1986        self.data[self.ct+1] = other
1987
1988    @property
1989    def coordinates(self):
1990        return tuple(self.data[self.cx:self.cx+3])
1991
1992    @property
1993    def occupancy(self):
1994        return self.data[self.cx+3]
1995
1996    @occupancy.setter
1997    def occupancy(self, val):
1998        self.data[self.cx+3] = float(val)
1999
2000    @property
2001    def ranId(self):
2002        return self.data[self.cia+8]
2003
2004    @property
2005    def adp_flag(self):
2006        # Either 'I' or 'A'
2007        return self.data[self.cia]
2008
2009    @property
2010    def uiso(self):
2011        if self.adp_flag == 'I':
2012            return self.data[self.cia+1]
2013        else:
2014            return self.data[self.cia+2:self.cia+8]
2015
2016    @uiso.setter
2017    def uiso(self, value):
2018        if self.adp_flag == 'I':
2019            self.data[self.cia+1] = float(value)
2020        else:
2021            assert len(value) == 6
2022            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2023
2024
2025class G2PwdrData(G2ObjectWrapper):
2026    """Wraps a Powder Data Histogram."""
2027    def __init__(self, data, proj):
2028        self.data = data
2029        self.proj = proj
2030
2031    @staticmethod
2032    def is_valid_refinement_key(key):
2033        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2034                      'Instrument Parameters']
2035        return key in valid_keys
2036
2037    @property
2038    def name(self):
2039        return self.data['data'][-1]
2040
2041    @property
2042    def ranId(self):
2043        return self.data['data'][0]['ranId']
2044
2045    @property
2046    def residuals(self):
2047        '''Provides a dictionary with with the R-factors for this histogram.
2048        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2049        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2050        '''
2051        data = self.data['data'][0]
2052        return {key: data[key] for key in data
2053                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2054                   or ':' in key}
2055
2056    @property
2057    def InstrumentParameters(self):
2058        '''Provides a dictionary with with the Instrument Parameters
2059        for this histogram.
2060        '''
2061        return self.data['Instrument Parameters'][0]
2062
2063    @property
2064    def SampleParameters(self):
2065        '''Provides a dictionary with with the Sample Parameters
2066        for this histogram.
2067        '''
2068        return self.data['Sample Parameters']
2069   
2070    @property
2071    def id(self):
2072        self.proj.update_ids()
2073        return self.data['data'][0]['hId']
2074
2075    @id.setter
2076    def id(self, val):
2077        self.data['data'][0]['hId'] = val
2078
2079    def fit_fixed_points(self):
2080        """Attempts to apply a background fit to the fixed points currently specified."""
2081        def SetInstParms(Inst):
2082            dataType = Inst['Type'][0]
2083            insVary = []
2084            insNames = []
2085            insVals = []
2086            for parm in Inst:
2087                insNames.append(parm)
2088                insVals.append(Inst[parm][1])
2089                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2090                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2091                        Inst[parm][2] = False
2092            instDict = dict(zip(insNames, insVals))
2093            instDict['X'] = max(instDict['X'], 0.01)
2094            instDict['Y'] = max(instDict['Y'], 0.01)
2095            if 'SH/L' in instDict:
2096                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2097            return dataType, instDict, insVary
2098
2099        bgrnd = self.data['Background']
2100
2101        # Need our fixed points in order
2102        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2103        X = [x for x, y in bgrnd[1]['FixedPoints']]
2104        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2105
2106        limits = self.data['Limits'][1]
2107        if X[0] > limits[0]:
2108            X = [limits[0]] + X
2109            Y = [Y[0]] + Y
2110        if X[-1] < limits[1]:
2111            X += [limits[1]]
2112            Y += [Y[-1]]
2113
2114        # Some simple lookups
2115        controls = self.proj['Controls']['data']
2116        inst, inst2 = self.data['Instrument Parameters']
2117        pwddata = self.data['data'][1]
2118
2119        # Construct the data for background fitting
2120        xBeg = np.searchsorted(pwddata[0], limits[0])
2121        xFin = np.searchsorted(pwddata[0], limits[1])
2122        xdata = pwddata[0][xBeg:xFin]
2123        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2124
2125        W = [1]*len(xdata)
2126        Z = [0]*len(xdata)
2127
2128        dataType, insDict, insVary = SetInstParms(inst)
2129        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2130
2131        # Do the fit
2132        data = np.array([xdata, ydata, W, Z, Z, Z])
2133        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2134                        prevVaryList=bakVary, controls=controls)
2135
2136        # Post-fit
2137        parmDict = {}
2138        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2139        parmDict.update(bakDict)
2140        parmDict.update(insDict)
2141        pwddata[3][xBeg:xFin] *= 0
2142        pwddata[5][xBeg:xFin] *= 0
2143        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2144
2145        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2146        # TODO update background
2147        self.proj.save()
2148
2149    def getdata(self,datatype):
2150        '''Provides access to the histogram data of the selected data type
2151
2152        :param str datatype: must be one of the following values (case is ignored)
2153       
2154           * 'X': the 2theta or TOF values for the pattern
2155           * 'Yobs': the observed intensity values
2156           * 'Yweight': the weights for each data point (1/sigma**2)
2157           * 'Ycalc': the computed intensity values
2158           * 'Background': the computed background values
2159           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2160
2161        :returns: an numpy MaskedArray with data values of the requested type
2162       
2163        '''
2164        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2165        if datatype.lower() not in enums:
2166            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2167        return self.data['data'][1][enums.index(datatype.lower())]
2168       
2169    def y_calc(self):
2170        return self.data['data'][1][3]
2171
2172    def Export(self,fileroot,extension):
2173        '''Write the histogram into a file. The path is specified by fileroot and
2174        extension.
2175       
2176        :param str fileroot: name of the file, optionally with a path (extension is
2177           ignored)
2178        :param str extension: includes '.', must match an extension in global
2179           exportersByExtension['powder'] or a Exception is raised.
2180        :returns: name of file that was written
2181        '''
2182        if extension not in exportersByExtension.get('powder',[]):
2183            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2184        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2185        obj = exportersByExtension['powder'][extension]
2186        obj.SetFromArray(hist=self.data,histname=self.name)
2187        obj.Writer(self.name,fil)
2188           
2189    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2190        try:
2191            import matplotlib.pyplot as plt
2192            data = self.data['data'][1]
2193            if Yobs:
2194                plt.plot(data[0], data[1], label='Yobs')
2195            if Ycalc:
2196                plt.plot(data[0], data[3], label='Ycalc')
2197            if Background:
2198                plt.plot(data[0], data[4], label='Background')
2199            if Residual:
2200                plt.plot(data[0], data[5], label="Residual")
2201        except ImportError:
2202            pass
2203
2204    def get_wR(self):
2205        """returns the overall weighted profile R factor for a histogram
2206       
2207        :returns: a wR value as a percentage or None if not defined
2208        """
2209        return self['data'][0].get('wR')
2210
2211    def set_refinements(self, refs):
2212        """Sets the refinement parameter 'key' to the specification 'value'
2213
2214        :param dict refs: A dictionary of the parameters to be set. See
2215                          :ref:`Histogram_parameters_table` for a description of
2216                          what these dictionaries should be.
2217
2218        :returns: None
2219
2220        """
2221        do_fit_fixed_points = False
2222        for key, value in refs.items():
2223            if key == 'Limits':
2224                old_limits = self.data['Limits'][1]
2225                new_limits = value
2226                if isinstance(new_limits, dict):
2227                    if 'low' in new_limits:
2228                        old_limits[0] = new_limits['low']
2229                    if 'high' in new_limits:
2230                        old_limits[1] = new_limits['high']
2231                else:
2232                    old_limits[0], old_limits[1] = new_limits
2233            elif key == 'Sample Parameters':
2234                sample = self.data['Sample Parameters']
2235                for sparam in value:
2236                    if sparam not in sample:
2237                        raise ValueError("Unknown refinement parameter, "
2238                                         + str(sparam))
2239                    sample[sparam][1] = True
2240            elif key == 'Background':
2241                bkg, peaks = self.data['Background']
2242
2243                # If True or False, just set the refine parameter
2244                if value in (True, False):
2245                    bkg[1] = value
2246                    return
2247
2248                if 'type' in value:
2249                    bkg[0] = value['type']
2250                if 'refine' in value:
2251                    bkg[1] = value['refine']
2252                if 'no. coeffs' in value:
2253                    cur_coeffs = bkg[2]
2254                    n_coeffs = value['no. coeffs']
2255                    if n_coeffs > cur_coeffs:
2256                        for x in range(n_coeffs - cur_coeffs):
2257                            bkg.append(0.0)
2258                    else:
2259                        for _ in range(cur_coeffs - n_coeffs):
2260                            bkg.pop()
2261                    bkg[2] = n_coeffs
2262                if 'coeffs' in value:
2263                    bkg[3:] = value['coeffs']
2264                if 'FixedPoints' in value:
2265                    peaks['FixedPoints'] = [(float(a), float(b))
2266                                            for a, b in value['FixedPoints']]
2267                if value.get('fit fixed points', False):
2268                    do_fit_fixed_points = True
2269
2270            elif key == 'Instrument Parameters':
2271                instrument, secondary = self.data['Instrument Parameters']
2272                for iparam in value:
2273                    try:
2274                        instrument[iparam][2] = True
2275                    except IndexError:
2276                        raise ValueError("Invalid key:", iparam)
2277            else:
2278                raise ValueError("Unknown key:", key)
2279        # Fit fixed points after the fact - ensure they are after fixed points
2280        # are added
2281        if do_fit_fixed_points:
2282            # Background won't be fit if refinement flag not set
2283            orig = self.data['Background'][0][1]
2284            self.data['Background'][0][1] = True
2285            self.fit_fixed_points()
2286            # Restore the previous value
2287            self.data['Background'][0][1] = orig
2288
2289    def clear_refinements(self, refs):
2290        """Clears the refinement parameter 'key' and its associated value.
2291
2292        :param dict refs: A dictionary of parameters to clear."""
2293        for key, value in refs.items():
2294            if key == 'Limits':
2295                old_limits, cur_limits = self.data['Limits']
2296                cur_limits[0], cur_limits[1] = old_limits
2297            elif key == 'Sample Parameters':
2298                sample = self.data['Sample Parameters']
2299                for sparam in value:
2300                    sample[sparam][1] = False
2301            elif key == 'Background':
2302                bkg, peaks = self.data['Background']
2303
2304                # If True or False, just set the refine parameter
2305                if value in (True, False):
2306                    bkg[1] = False
2307                    return
2308
2309                bkg[1] = False
2310                if 'FixedPoints' in value:
2311                    if 'FixedPoints' in peaks:
2312                        del peaks['FixedPoints']
2313            elif key == 'Instrument Parameters':
2314                instrument, secondary = self.data['Instrument Parameters']
2315                for iparam in value:
2316                    instrument[iparam][2] = False
2317            else:
2318                raise ValueError("Unknown key:", key)
2319
2320    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2321        '''Adds a single peak to the peak list
2322        :param float area: peak area
2323        :param float dspace: peak position as d-space (A)
2324        :param float Q: peak position as Q (A-1)
2325        :param float ttheta: peak position as 2Theta (deg)
2326
2327        Note: only one of the parameters dspace, Q or ttheta may be specified
2328        '''
2329        import GSASIIlattice as G2lat
2330        import GSASIImath as G2mth
2331        if (not dspace) + (not Q) + (not ttheta) != 2:
2332            print('add_peak error: too many or no peak position(s) specified')
2333            return
2334        pos = ttheta
2335        Parms,Parms2 = self.data['Instrument Parameters']
2336        if Q:
2337            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2338        elif dspace:
2339            pos = G2lat.Dsp2pos(Parms,dspace)
2340        peaks = self.data['Peak List']
2341        peaks['sigDict'] = {}        #no longer valid
2342        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2343
2344    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2345        '''Set refinement flags for peaks
2346        :param list peaklist: a list of peaks to change flags. If None (default), changes
2347          are made to all peaks.
2348        :param bool area: Sets or clears the refinement flag for the peak area value.
2349          If None (the default), no change is made.
2350        :param bool pos: Sets or clears the refinement flag for the peak position value.
2351          If None (the default), no change is made.
2352        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2353          If None (the default), no change is made.
2354        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2355          If None (the default), no change is made.
2356         
2357        Note that when peaks are first created the area flag is on and the other flags are
2358        initially off.
2359
2360        Example::
2361       
2362           set_peakFlags(sig=False,gam=True)
2363
2364        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
2365        all peaks. The position and area flags are not changed from their previous values.
2366        '''
2367        peaks = self.data['Peak List']
2368        if peaklist is None:
2369            peaklist = range(len(peaks['peaks']))
2370        for i in peaklist:
2371            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
2372                if var is not None:
2373                    peaks['peaks'][i][j] = var
2374           
2375    def refine_peaks(self):
2376        '''Causes a refinement of peak position, background and instrument parameters
2377        '''
2378        import GSASIIpwd as G2pwd
2379        controls = self.proj.data.get('Controls',{})
2380        controls = controls.get('data',
2381                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
2382                            )
2383        peaks = self.data['Peak List']
2384        Parms,Parms2 = self.data['Instrument Parameters']
2385        background = self.data['Background']
2386        limits = self.data['Limits'][1]
2387        bxye = np.zeros(len(self.data['data'][1][1]))
2388        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
2389                                           Parms,Parms2,self.data['data'][1],bxye,[],
2390                                           False,controls,None)[0]
2391
2392    def SaveProfile(self,filename):
2393        '''Writes a GSAS-II (new style) .instprm file
2394        '''
2395        data,Parms2 = self.data['Instrument Parameters']
2396        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2397        File = open(filename,'w')
2398        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
2399        for item in data:
2400            File.write(item+':'+str(data[item][1])+'\n')
2401        File.close()
2402        print ('Instrument parameters saved to: '+filename)
2403
2404    def LoadProfile(self,filename):
2405        '''Reads a GSAS-II (new style) .instprm file and overwrites the current parameters
2406        '''
2407        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2408        File = open(filename,'r')
2409        S = File.readline()
2410        newItems = []
2411        newVals = []
2412        Found = False
2413        while S:
2414            if S[0] == '#':
2415                if Found:
2416                    break
2417                if 'Bank' in S:
2418                    if bank == int(S.split(':')[0].split()[1]):
2419                        S = File.readline()
2420                        continue
2421                    else:
2422                        S = File.readline()
2423                        while S and '#Bank' not in S:
2424                            S = File.readline()
2425                        continue
2426                else:   #a non #Bank file
2427                    S = File.readline()
2428                    continue
2429            Found = True
2430            [item,val] = S[:-1].split(':')
2431            newItems.append(item)
2432            try:
2433                newVals.append(float(val))
2434            except ValueError:
2435                newVals.append(val)                       
2436            S = File.readline()               
2437        File.close()
2438        LoadG2fil()
2439        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
2440
2441       
2442class G2Phase(G2ObjectWrapper):
2443    """A wrapper object around a given phase.
2444
2445    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
2446    """
2447    def __init__(self, data, name, proj):
2448        self.data = data
2449        self.name = name
2450        self.proj = proj
2451
2452    @staticmethod
2453    def is_valid_refinement_key(key):
2454        valid_keys = ["Cell", "Atoms", "LeBail"]
2455        return key in valid_keys
2456
2457    @staticmethod
2458    def is_valid_HAP_refinement_key(key):
2459        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
2460                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
2461        return key in valid_keys
2462
2463    def atom(self, atomlabel):
2464        """Returns the atom specified by atomlabel, or None if it does not
2465        exist.
2466
2467        :param str atomlabel: The name of the atom (e.g. "O2")
2468        :returns: A :class:`G2AtomRecord` object
2469            representing the atom.
2470        """
2471        # Consult GSASIIobj.py for the meaning of this
2472        cx, ct, cs, cia = self.data['General']['AtomPtrs']
2473        ptrs = [cx, ct, cs, cia]
2474        atoms = self.data['Atoms']
2475        for atom in atoms:
2476            if atom[ct-1] == atomlabel:
2477                return G2AtomRecord(atom, ptrs, self.proj)
2478
2479    def atoms(self):
2480        """Returns a list of atoms present in the phase.
2481
2482        :returns: A list of :class:`G2AtomRecord` objects.
2483
2484        .. seealso::
2485            :meth:`G2Phase.atom`
2486            :class:`G2AtomRecord`
2487        """
2488        ptrs = self.data['General']['AtomPtrs']
2489        output = []
2490        atoms = self.data['Atoms']
2491        for atom in atoms:
2492            output.append(G2AtomRecord(atom, ptrs, self.proj))
2493        return output
2494
2495    def histograms(self):
2496        output = []
2497        for hname in self.data.get('Histograms', {}).keys():
2498            output.append(self.proj.histogram(hname))
2499        return output
2500
2501    @property
2502    def ranId(self):
2503        return self.data['ranId']
2504
2505    @property
2506    def id(self):
2507        return self.data['pId']
2508
2509    @id.setter
2510    def id(self, val):
2511        self.data['pId'] = val
2512
2513    def get_cell(self):
2514        """Returns a dictionary of the cell parameters, with keys:
2515            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
2516
2517        :returns: a dict
2518
2519        .. seealso::
2520           :meth:`G2Phase.get_cell_and_esd`
2521
2522        """
2523        cell = self.data['General']['Cell']
2524        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
2525                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
2526                'volume': cell[7]}
2527
2528    def get_cell_and_esd(self):
2529        """
2530        Returns a pair of dictionaries, the first representing the unit cell, the second
2531        representing the estimated standard deviations of the unit cell.
2532
2533        :returns: a tuple of two dictionaries
2534
2535        .. seealso::
2536           :meth:`G2Phase.get_cell`
2537
2538        """
2539        # translated from GSASIIstrIO.ExportBaseclass.GetCell
2540        import GSASIIstrIO as G2stIO
2541        import GSASIIlattice as G2lat
2542        import GSASIImapvars as G2mv
2543        try:
2544            pfx = str(self.id) + '::'
2545            sgdata = self['General']['SGData']
2546            covDict = self.proj['Covariance']['data']
2547
2548            parmDict = dict(zip(covDict.get('varyList',[]),
2549                                covDict.get('variables',[])))
2550            sigDict = dict(zip(covDict.get('varyList',[]),
2551                               covDict.get('sig',[])))
2552
2553            if covDict.get('covMatrix') is not None:
2554                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2555                                                  covDict['varyList'],
2556                                                  parmDict))
2557
2558            A, sigA = G2stIO.cellFill(pfx, sgdata, parmDict, sigDict)
2559            cellSig = G2stIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
2560            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
2561            cellDict, cellSigDict = {}, {}
2562            for i, key in enumerate(['length_a', 'length_b', 'length_c',
2563                                     'angle_alpha', 'angle_beta', 'angle_gamma',
2564                                     'volume']):
2565                cellDict[key] = cellList[i]
2566                cellSigDict[key] = cellSig[i]
2567            return cellDict, cellSigDict
2568        except KeyError:
2569            cell = self.get_cell()
2570            return cell, {key: 0.0 for key in cell}
2571
2572    def export_CIF(self, outputname, quickmode=True):
2573        """Write this phase to a .cif file named outputname
2574
2575        :param str outputname: The name of the .cif file to write to
2576        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
2577        # This code is all taken from exports/G2export_CIF.py
2578        # Functions copied have the same names
2579        import GSASIImath as G2mth
2580        import GSASIImapvars as G2mv
2581        from exports import G2export_CIF as cif
2582
2583        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
2584        CIFname = os.path.splitext(self.proj.filename)[0]
2585        CIFname = os.path.split(CIFname)[1]
2586        CIFname = ''.join([c if ord(c) < 128 else ''
2587                           for c in CIFname.replace(' ', '_')])
2588        try:
2589            author = self.proj['Controls']['data'].get('Author','').strip()
2590        except KeyError:
2591            pass
2592        oneblock = True
2593
2594        covDict = self.proj['Covariance']['data']
2595        parmDict = dict(zip(covDict.get('varyList',[]),
2596                            covDict.get('variables',[])))
2597        sigDict = dict(zip(covDict.get('varyList',[]),
2598                           covDict.get('sig',[])))
2599
2600        if covDict.get('covMatrix') is not None:
2601            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2602                                              covDict['varyList'],
2603                                              parmDict))
2604
2605        with open(outputname, 'w') as fp:
2606            fp.write(' \n' + 70*'#' + '\n')
2607            cif.WriteCIFitem(fp, 'data_' + CIFname)
2608            # from exports.G2export_CIF.WritePhaseInfo
2609            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
2610            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
2611            # TODO get esds
2612            cellDict = self.get_cell()
2613            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
2614            names = ['length_a','length_b','length_c',
2615                     'angle_alpha','angle_beta ','angle_gamma',
2616                     'volume']
2617            for key, val in cellDict.items():
2618                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
2619
2620            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
2621                         self.data['General']['SGData']['SGSys'])
2622
2623            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
2624            # regularize capitalization and remove trailing H/R
2625            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
2626            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
2627
2628            # generate symmetry operations including centering and center of symmetry
2629            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
2630                self.data['General']['SGData'])
2631            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
2632            for i, op in enumerate(SymOpList,start=1):
2633                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
2634
2635            # TODO skipped histograms, exports/G2export_CIF.py:880
2636
2637            # report atom params
2638            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
2639                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
2640                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
2641            else:
2642                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
2643            # report cell contents
2644            cif.WriteComposition(fp, self.data, self.name, parmDict)
2645            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
2646                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
2647                raise NotImplementedError("only quickmode currently supported")
2648            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
2649                cif.WriteCIFitem(fp,'\n# Difference density results')
2650                MinMax = self.data['General']['Map']['minmax']
2651                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
2652                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
2653
2654
2655    def set_refinements(self, refs):
2656        """Sets the refinement parameter 'key' to the specification 'value'
2657
2658        :param dict refs: A dictionary of the parameters to be set. See
2659                          :ref:`Phase_parameters_table` for a description of
2660                          this dictionary.
2661
2662        :returns: None"""
2663        for key, value in refs.items():
2664            if key == "Cell":
2665                self.data['General']['Cell'][0] = value
2666
2667            elif key == "Atoms":
2668                for atomlabel, atomrefinement in value.items():
2669                    if atomlabel == 'all':
2670                        for atom in self.atoms():
2671                            atom.refinement_flags = atomrefinement
2672                    else:
2673                        atom = self.atom(atomlabel)
2674                        if atom is None:
2675                            raise ValueError("No such atom: " + atomlabel)
2676                        atom.refinement_flags = atomrefinement
2677
2678            elif key == "LeBail":
2679                hists = self.data['Histograms']
2680                for hname, hoptions in hists.items():
2681                    if 'LeBail' not in hoptions:
2682                        hoptions['newLeBail'] = bool(True)
2683                    hoptions['LeBail'] = bool(value)
2684            else:
2685                raise ValueError("Unknown key:", key)
2686
2687    def clear_refinements(self, refs):
2688        """Clears a given set of parameters.
2689
2690        :param dict refs: The parameters to clear"""
2691        for key, value in refs.items():
2692            if key == "Cell":
2693                self.data['General']['Cell'][0] = False
2694            elif key == "Atoms":
2695                cx, ct, cs, cia = self.data['General']['AtomPtrs']
2696
2697                for atomlabel in value:
2698                    atom = self.atom(atomlabel)
2699                    # Set refinement to none
2700                    atom.refinement_flags = ' '
2701            elif key == "LeBail":
2702                hists = self.data['Histograms']
2703                for hname, hoptions in hists.items():
2704                    if 'LeBail' not in hoptions:
2705                        hoptions['newLeBail'] = True
2706                    hoptions['LeBail'] = False
2707            else:
2708                raise ValueError("Unknown key:", key)
2709
2710    def set_HAP_refinements(self, refs, histograms='all'):
2711        """Sets the given HAP refinement parameters between this phase and
2712        the given histograms
2713
2714        :param dict refs: A dictionary of the parameters to be set. See
2715                          :ref:`HAP_parameters_table` for a description of this
2716                          dictionary.
2717        :param histograms: Either 'all' (default) or a list of the histograms
2718            whose HAP parameters will be set with this phase. Histogram and phase
2719            must already be associated
2720
2721        :returns: None
2722        """
2723        if histograms == 'all':
2724            histograms = self.data['Histograms'].values()
2725        else:
2726            histograms = [self.data['Histograms'][h.name] for h in histograms]
2727
2728        for key, val in refs.items():
2729            for h in histograms:
2730                if key == 'Babinet':
2731                    try:
2732                        sets = list(val)
2733                    except ValueError:
2734                        sets = ['BabA', 'BabU']
2735                    for param in sets:
2736                        if param not in ['BabA', 'BabU']:
2737                            raise ValueError("Not sure what to do with" + param)
2738                        for hist in histograms:
2739                            hist['Babinet'][param][1] = True
2740                elif key == 'Extinction':
2741                    for h in histograms:
2742                        h['Extinction'][1] = bool(val)
2743                elif key == 'HStrain':
2744                    for h in histograms:
2745                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
2746                elif key == 'Mustrain':
2747                    for h in histograms:
2748                        mustrain = h['Mustrain']
2749                        newType = None
2750                        direction = None
2751                        if isinstance(val, strtypes):
2752                            if val in ['isotropic', 'uniaxial', 'generalized']:
2753                                newType = val
2754                            else:
2755                                raise ValueError("Not a Mustrain type: " + val)
2756                        elif isinstance(val, dict):
2757                            newType = val.get('type', None)
2758                            direction = val.get('direction', None)
2759
2760                        if newType:
2761                            mustrain[0] = newType
2762                            if newType == 'isotropic':
2763                                mustrain[2][0] = True
2764                                mustrain[5] = [False for p in mustrain[4]]
2765                            elif newType == 'uniaxial':
2766                                if 'refine' in val:
2767                                    types = val['refine']
2768                                    if isinstance(types, strtypes):
2769                                        types = [types]
2770                                    elif isinstance(types, bool):
2771                                        mustrain[2][0] = types
2772                                        mustrain[2][1] = types
2773                                        types = []
2774                                    else:
2775                                        raise ValueError("Not sure what to do with: "
2776                                                         + str(types))
2777                                else:
2778                                    types = []
2779
2780                                for unitype in types:
2781                                    if unitype == 'equatorial':
2782                                        mustrain[2][0] = True
2783                                    elif unitype == 'axial':
2784                                        mustrain[2][1] = True
2785                                    else:
2786                                        msg = 'Invalid uniaxial mustrain type'
2787                                        raise ValueError(msg + ': ' + unitype)
2788                            else:  # newtype == 'generalized'
2789                                mustrain[2] = [False for p in mustrain[1]]
2790
2791                        if direction:
2792                            if len(direction) != 3:
2793                                raise ValueError("Expected hkl, found", direction)
2794                            direction = [int(n) for n in direction]
2795                            mustrain[3] = direction
2796                elif key == 'Size':
2797                    for h in histograms:
2798                        size = h['Size']
2799                        newType = None
2800                        direction = None
2801                        if isinstance(val, strtypes):
2802                            if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
2803                                newType = val
2804                            else:
2805                                raise ValueError("Not a valid Size type: " + val)
2806                        elif isinstance(val, dict):
2807                            newType = val.get('type', None)
2808                            direction = val.get('direction', None)
2809
2810                        if newType:
2811                            size[0] = newType
2812                            refine = val.get('refine')
2813                            if newType == 'isotropic' and refine is not None:
2814                                size[2][0] = bool(refine)
2815                            elif newType == 'uniaxial' and refine is not None:
2816                                size[2][1] = bool(refine)
2817                                size[2][2] = bool(refine)
2818                            elif newType == 'ellipsoidal' and refine is not None:
2819                                size[5] = [bool(refine) for p in size[5]]
2820
2821                        if direction:
2822                            if len(direction) != 3:
2823                                raise ValueError("Expected hkl, found", direction)
2824                            direction = [int(n) for n in direction]
2825                            size[3] = direction
2826                elif key == 'Pref.Ori.':
2827                    for h in histograms:
2828                        h['Pref.Ori.'][2] = bool(val)
2829                elif key == 'Show':
2830                    for h in histograms:
2831                        h['Show'] = bool(val)
2832                elif key == 'Use':
2833                    for h in histograms:
2834                        h['Use'] = bool(val)
2835                elif key == 'Scale':
2836                    for h in histograms:
2837                        h['Scale'][1] = bool(val)
2838                else:
2839                    print(u'Unknown HAP key: '+key)
2840
2841    def getHAPvalues(self, histname):
2842        """Returns a dict with HAP values for the selected histogram
2843
2844        :param histogram: is a histogram object (:class:`G2PwdrData`) or
2845            a histogram name or the index number of the histogram
2846
2847        :returns: HAP value dict
2848        """
2849        if isinstance(histname, G2PwdrData):
2850            histname = histname.name
2851        elif histname in self.data['Histograms']:
2852            pass
2853        elif type(histname) is int:
2854            histname = self.proj.histograms()[histname].name
2855        else:
2856            raise G2ScriptException("Invalid histogram reference: "+str(histname))
2857        return self.data['Histograms'][histname]
2858                   
2859    def clear_HAP_refinements(self, refs, histograms='all'):
2860        """Clears the given HAP refinement parameters between this phase and
2861        the given histograms
2862
2863        :param dict refs: A dictionary of the parameters to be cleared.
2864        :param histograms: Either 'all' (default) or a list of the histograms
2865            whose HAP parameters will be cleared with this phase. Histogram and
2866            phase must already be associated
2867
2868        :returns: None
2869        """
2870        if histograms == 'all':
2871            histograms = self.data['Histograms'].values()
2872        else:
2873            histograms = [self.data['Histograms'][h.name] for h in histograms]
2874
2875        for key, val in refs.items():
2876            for h in histograms:
2877                if key == 'Babinet':
2878                    try:
2879                        sets = list(val)
2880                    except ValueError:
2881                        sets = ['BabA', 'BabU']
2882                    for param in sets:
2883                        if param not in ['BabA', 'BabU']:
2884                            raise ValueError("Not sure what to do with" + param)
2885                        for hist in histograms:
2886                            hist['Babinet'][param][1] = False
2887                elif key == 'Extinction':
2888                    for h in histograms:
2889                        h['Extinction'][1] = False
2890                elif key == 'HStrain':
2891                    for h in histograms:
2892                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
2893                elif key == 'Mustrain':
2894                    for h in histograms:
2895                        mustrain = h['Mustrain']
2896                        mustrain[2] = [False for p in mustrain[2]]
2897                        mustrain[5] = [False for p in mustrain[4]]
2898                elif key == 'Pref.Ori.':
2899                    for h in histograms:
2900                        h['Pref.Ori.'][2] = False
2901                elif key == 'Show':
2902                    for h in histograms:
2903                        h['Show'] = False
2904                elif key == 'Size':
2905                    for h in histograms:
2906                        size = h['Size']
2907                        size[2] = [False for p in size[2]]
2908                        size[5] = [False for p in size[5]]
2909                elif key == 'Use':
2910                    for h in histograms:
2911                        h['Use'] = False
2912                elif key == 'Scale':
2913                    for h in histograms:
2914                        h['Scale'][1] = False
2915                else:
2916                    print(u'Unknown HAP key: '+key)
2917
2918
2919##########################
2920# Command Line Interface #
2921##########################
2922# Each of these takes an argparse.Namespace object as their argument,
2923# representing the parsed command-line arguments for the relevant subcommand.
2924# The argument specification for each is in the subcommands dictionary (see
2925# below)
2926
2927commandhelp={}
2928commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
2929def create(args):
2930    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
2931
2932  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2933                                  [-i IPARAMS [IPARAMS ...]]
2934                                  [-p PHASES [PHASES ...]]
2935                                  filename
2936                                 
2937positional arguments::
2938
2939  filename              the project file to create. should end in .gpx
2940
2941optional arguments::
2942
2943  -h, --help            show this help message and exit
2944  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2945                        list of datafiles to add as histograms
2946  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2947                        instrument parameter file, must be one for every
2948                        histogram
2949  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2950                        list of phases to add. phases are automatically
2951                        associated with all histograms given.
2952
2953    """
2954    proj = G2Project(gpxname=args.filename)
2955
2956    hist_objs = []
2957    if args.histograms:
2958        for h,i in zip(args.histograms,args.iparams):
2959            print("Adding histogram from",h,"with instparm ",i)
2960            hist_objs.append(proj.add_powder_histogram(h, i))
2961
2962    if args.phases: 
2963        for p in args.phases:
2964            print("Adding phase from",p)
2965            proj.add_phase(p, histograms=hist_objs)
2966        print('Linking phase(s) to histogram(s):')
2967        for h in hist_objs:
2968            print ('   '+h.name)
2969
2970    proj.save()
2971
2972commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
2973def add(args):
2974    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
2975
2976  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2977                               [-i IPARAMS [IPARAMS ...]]
2978                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
2979                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
2980                               filename
2981
2982
2983positional arguments::
2984
2985  filename              the project file to open. Should end in .gpx
2986
2987optional arguments::
2988
2989  -h, --help            show this help message and exit
2990  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2991                        list of datafiles to add as histograms
2992  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2993                        instrument parameter file, must be one for every
2994                        histogram
2995  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
2996                        format hint for histogram import. Applies to all
2997                        histograms
2998  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2999                        list of phases to add. phases are automatically
3000                        associated with all histograms given.
3001  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
3002                        format hint for phase import. Applies to all phases.
3003                        Example: -pf CIF
3004  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
3005                        list of histgram indices to associate with added
3006                        phases. If not specified, phases are associated with
3007                        all previously loaded histograms. Example: -l 2 3 4
3008   
3009    """
3010    proj = G2Project(args.filename)
3011
3012    if args.histograms:
3013        for h,i in zip(args.histograms,args.iparams):
3014            print("Adding histogram from",h,"with instparm ",i)
3015            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
3016
3017    if args.phases: 
3018        if not args.histlist:
3019            histlist = proj.histograms()
3020        else:
3021            histlist = [proj.histogram(i) for i in args.histlist]
3022
3023        for p in args.phases:
3024            print("Adding phase from",p)
3025            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
3026           
3027        if not args.histlist:
3028            print('Linking phase(s) to all histogram(s)')
3029        else:
3030            print('Linking phase(s) to histogram(s):')
3031            for h in histlist:
3032                print ('   '+h.name)
3033
3034    proj.save()
3035
3036
3037commandhelp["dump"] = "Shows the contents of a GSAS-II project"
3038def dump(args):
3039    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
3040
3041       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
3042
3043positional arguments::
3044
3045  files
3046
3047optional arguments::
3048
3049  -h, --help        show this help message and exit
3050  -d, --histograms  list histograms in files, overrides --raw
3051  -p, --phases      list phases in files, overrides --raw
3052  -r, --raw         dump raw file contents, default
3053 
3054    """
3055    if not args.histograms and not args.phases:
3056        args.raw = True
3057    if args.raw:
3058        import IPython.lib.pretty as pretty
3059
3060    for fname in args.files:
3061        if args.raw:
3062            proj, nameList = LoadDictFromProjFile(fname)
3063            print("file:", fname)
3064            print("names:", nameList)
3065            for key, val in proj.items():
3066                print(key, ":")
3067                pretty.pprint(val)
3068        else:
3069            proj = G2Project(fname)
3070            if args.histograms:
3071                hists = proj.histograms()
3072                for h in hists:
3073                    print(fname, "hist", h.id, h.name)
3074            if args.phases:
3075                phase_list = proj.phases()
3076                for p in phase_list:
3077                    print(fname, "phase", p.id, p.name)
3078
3079
3080commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
3081def IPyBrowse(args):
3082    """Load a .gpx file and then open a IPython shell to browse it::
3083
3084  usage: GSASIIscriptable.py browse [-h] files [files ...]
3085
3086positional arguments::
3087
3088  files       list of files to browse
3089
3090optional arguments::
3091
3092  -h, --help  show this help message and exit
3093
3094    """
3095    for fname in args.files:
3096        proj, nameList = LoadDictFromProjFile(fname)
3097        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
3098        GSASIIpath.IPyBreak_base(msg)
3099        break
3100
3101
3102commandhelp["refine"] = '''
3103Conducts refinements on GSAS-II projects according to a list of refinement
3104steps in a JSON dict
3105'''
3106def refine(args):
3107    """Implements the refine command-line subcommand:
3108    conducts refinements on GSAS-II projects according to a JSON refinement dict::
3109
3110        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
3111
3112positional arguments::
3113
3114  gpxfile      the project file to refine
3115  refinements  json file of refinements to apply. if not present refines file
3116               as-is
3117
3118optional arguments::
3119
3120  -h, --help   show this help message and exit
3121 
3122    """
3123    proj = G2Project(args.gpxfile)
3124    if args.refinements is None:
3125        proj.refine()
3126    else:
3127        import json
3128        with open(args.refinements) as refs:
3129            refs = json.load(refs)
3130        if type(refs) is not dict:
3131            raise G2ScriptException("Error: JSON object must be a dict.")
3132        if "code" in refs:
3133            print("executing code:\n|  ",'\n|  '.join(refs['code']))
3134            exec('\n'.join(refs['code']))
3135        proj.do_refinements(refs['refinements'])
3136
3137
3138commandhelp["seqrefine"] = "Not implemented. Placeholder for eventual sequential refinement implementation"
3139def seqrefine(args):
3140    """Future implementation for the seqrefine command-line subcommand """
3141    raise NotImplementedError("seqrefine is not yet implemented")
3142
3143
3144commandhelp["export"] = "Export phase as CIF"
3145def export(args):
3146    """Implements the export command-line subcommand: Exports phase as CIF::
3147
3148      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
3149
3150positional arguments::
3151
3152  gpxfile     the project file from which to export
3153  phase       identifier of phase to export
3154  exportfile  the .cif file to export to
3155
3156optional arguments::
3157
3158  -h, --help  show this help message and exit
3159
3160    """
3161    proj = G2Project(args.gpxfile)
3162    phase = proj.phase(args.phase)
3163    phase.export_CIF(args.exportfile)
3164
3165
3166def _args_kwargs(*args, **kwargs):
3167    return args, kwargs
3168
3169# A dictionary of the name of each subcommand, and a tuple
3170# of its associated function and a list of its arguments
3171# The arguments are passed directly to the add_argument() method
3172# of an argparse.ArgumentParser
3173
3174subcommands = {"create":
3175               (create, [_args_kwargs('filename',
3176                                      help='the project file to create. should end in .gpx'),
3177
3178                         _args_kwargs('-d', '--histograms',
3179                                      nargs='+',
3180                                      help='list of datafiles to add as histograms'),
3181                                     
3182                         _args_kwargs('-i', '--iparams',
3183                                      nargs='+',
3184                                      help='instrument parameter file, must be one'
3185                                           ' for every histogram'
3186                                      ),
3187
3188                         _args_kwargs('-p', '--phases',
3189                                      nargs='+',
3190                                      help='list of phases to add. phases are '
3191                                           'automatically associated with all '
3192                                           'histograms given.')]),
3193               "add": (add, [_args_kwargs('filename',
3194                                      help='the project file to open. Should end in .gpx'),
3195
3196                         _args_kwargs('-d', '--histograms',
3197                                      nargs='+',
3198                                      help='list of datafiles to add as histograms'),
3199                                     
3200                         _args_kwargs('-i', '--iparams',
3201                                      nargs='+',
3202                                      help='instrument parameter file, must be one'
3203                                           ' for every histogram'
3204                                      ),
3205                                     
3206                         _args_kwargs('-hf', '--histogramformat',
3207                                      help='format hint for histogram import. Applies to all'
3208                                           ' histograms'
3209                                      ),
3210
3211                         _args_kwargs('-p', '--phases',
3212                                      nargs='+',
3213                                      help='list of phases to add. phases are '
3214                                           'automatically associated with all '
3215                                           'histograms given.'),
3216
3217                         _args_kwargs('-pf', '--phaseformat',
3218                                      help='format hint for phase import. Applies to all'
3219                                           ' phases. Example: -pf CIF'
3220                                      ),
3221                                     
3222                         _args_kwargs('-l', '--histlist',
3223                                      nargs='+',
3224                                      help='list of histgram indices to associate with added'
3225                                           ' phases. If not specified, phases are'
3226                                           ' associated with all previously loaded'
3227                                           ' histograms. Example: -l 2 3 4')]),
3228                                           
3229               "dump": (dump, [_args_kwargs('-d', '--histograms',
3230                                     action='store_true',
3231                                     help='list histograms in files, overrides --raw'),
3232
3233                               _args_kwargs('-p', '--phases',
3234                                            action='store_true',
3235                                            help='list phases in files, overrides --raw'),
3236
3237                               _args_kwargs('-r', '--raw',
3238                                      action='store_true', help='dump raw file contents, default'),
3239
3240                               _args_kwargs('files', nargs='+')]),
3241
3242               "refine":
3243               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
3244                         _args_kwargs('refinements',
3245                                      help='JSON file of refinements to apply. if not present'
3246                                           ' refines file as-is',
3247                                      default=None,
3248                                      nargs='?')]),
3249
3250               "seqrefine": (seqrefine, []),
3251               "export": (export, [_args_kwargs('gpxfile',
3252                                                help='the project file from which to export'),
3253                                   _args_kwargs('phase', help='identifier of phase to export'),
3254                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
3255               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
3256                                                   help='list of files to browse')])}
3257
3258
3259def main():
3260    '''The command-line interface for calling GSASIIscriptable as a shell command,
3261    where it is expected to be called as::
3262
3263       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
3264
3265    The following subcommands are defined:
3266
3267        * create, see :func:`create`
3268        * add, see :func:`add`
3269        * dump, see :func:`dump`
3270        * refine, see :func:`refine`
3271        * seqrefine, see :func:`seqrefine`
3272        * export, :func:`export`
3273        * browse, see :func:`IPyBrowse`
3274
3275    .. seealso::
3276        :func:`create`
3277        :func:`add`
3278        :func:`dump`
3279        :func:`refine`
3280        :func:`seqrefine`
3281        :func:`export`
3282        :func:`IPyBrowse`
3283    '''
3284    parser = argparse.ArgumentParser(description=
3285        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
3286        )
3287    subs = parser.add_subparsers()
3288
3289    # Create all of the specified subparsers
3290    for name, (func, args) in subcommands.items():
3291        new_parser = subs.add_parser(name,help=commandhelp.get(name),
3292                                     description='Command "'+name+'" '+commandhelp.get(name))
3293        for listargs, kwds in args:
3294            new_parser.add_argument(*listargs, **kwds)
3295        new_parser.set_defaults(func=func)
3296
3297    # Parse and trigger subcommand
3298    result = parser.parse_args()
3299    result.func(result)
3300
3301if __name__ == '__main__':
3302    main()
Note: See TracBrowser for help on using the repository browser.