source: trunk/GSASIIscriptable.py @ 3504

Last change on this file since 3504 was 3504, checked in by svnjenkins, 3 years ago

scripting: fix mustrain refine use; add size value

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 133.7 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2018-07-24 03:51:15 +0000 (Tue, 24 Jul 2018) $
5# $Author: svnjenkins $
6# $Revision: 3504 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3504 2018-07-24 03:51:15Z svnjenkins $
9########### SVN repository information ###################
10#
11# TODO: integrate 2d data based on a model
12#
13"""
14*GSASIIscriptable: Scripting Interface*
15=======================================
16
17Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
18This file specifies several wrapper classes around GSAS-II data representations.
19They all inherit from :class:`G2ObjectWrapper`. The chief class is :class:`G2Project`,
20which represents an entire GSAS-II project and provides several methods to access
21phases, powder histograms, and execute Rietveld refinements. These routines can be
22accessed in two ways:
23the :ref:`CommandlineInterface` provides access a number of features without writing
24Python scripts of from the :ref:`API`, which allows much more versatile access from
25within Python.
26
27
28=====================
29Overview
30=====================
31The most commonly used routines are:
32
33:class:`G2Project`
34   Always needed: used to create a new project (.gpx) file or read in an existing one.
35
36:meth:`G2Project.add_powder_histogram`
37   Used to read in powder diffraction data to a project file.
38
39:meth:`G2Project.add_simulated_powder_histogram`
40   Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
41
42:meth:`G2Project.save`
43   Writes the current project to disk.
44
45:meth:`G2Project.do_refinements`
46   This is passed a list of dictionaries, where each dict defines a refinement step.
47   Passing a list with a single empty dict initiates a refinement with the current
48   parameters and flags.
49   
50:meth:`G2Project.set_refinement`
51   This is passed a single dict which is used to set parameters and flags.
52   These actions can be performed also in :meth:`G2Project.do_refinements`.
53
54Refinement dicts
55   A refinement dict sets up a single refinement step
56   (as used in :meth:`G2Project.do_refinements` and described in
57   :ref:`Project_dicts`).
58   It specifies parameter & refinement flag changes, which are usually followed by a refinement and
59   optionally by calls to locally-defined Python functions. The keys used in these dicts are
60   defined in the :ref:`Refinement_recipe` table, below.
61
62There are several ways to set parameters project using different level objects, as is
63are described in sections below, but the simplest way to access parameters and flags
64in a project is to use the above routines.
65
66=====================
67Refinement parameters
68=====================
69The most complex part of scripting GSAS-II comes in setting up the input to control refinements, which is
70described here.
71
72.. _Project_dicts:
73
74-----------------------------
75Project-level Parameter Dict
76-----------------------------
77
78As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
79which can be accessed individually by the objects that encapsulate individual phases and histograms
80but it will be usually simplest to create a composite dictionary
81that is used at the project-level. A dict is created with keys
82"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
83(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
84determine parameter values and will determine which parameters will be refined.
85
86The specific keys and subkeys that can be used are defined in tables
87:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
88
89Note that optionally a list of histograms and/or phases can be supplied in the call to
90:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
91phases and histograms.
92
93As an example:
94
95.. code-block::  python
96
97    pardict = {'set': { 'Limits': [0.8, 12.0],
98                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
99                       'Background': {'type': 'chebyschev', 'refine': True}},
100              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
101    my_project.set_refinement(pardict)
102   
103.. _Refinement_recipe:
104   
105------------------------
106Refinement recipe
107------------------------
108Building on the :ref:`Project_dicts`,
109it is possible to specify a sequence of refinement actions as a list of
110these dicts and supplying this list
111as an argument to :meth:`G2Project.do_refinements`.
112
113As an example, this code performs the same actions as in the example in the section above:
114
115.. code-block::  python
116   
117    pardict = {'set': { 'Limits': [0.8, 12.0],
118                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
119                       'Background': {'type': 'chebyschev', 'refine': True}},
120              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
121    my_project.do_refinements([pardict])
122
123However, in addition to setting a number of parameters, this example will perform a refinement as well.
124If more than one dict is specified in the list, as is done in this example,
125two refinement steps will be performed:
126
127.. code-block::  python
128
129    my_project.do_refinements([pardict,pardict1])
130
131
132The keys defined in the following table
133may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that now keys ``histograms``
134and ``phases`` are used to limit actions to specific sets of parameters within the project.
135
136========== ============================================================================
137key         explanation
138========== ============================================================================
139set                    Specifies a dict with keys and subkeys as described in the
140                       :ref:`Refinement_parameters_fmt` section. Items listed here
141                       will be set to be refined.
142clear                  Specifies a dict, as above for set, except that parameters are
143                       cleared and thus will not be refined.
144once                   Specifies a dict as above for set, except that parameters are
145                       set for the next cycle of refinement and are cleared once the
146                       refinement step is completed.
147skip                   Normally, once parameters are processed with a set/clear/once
148                       action(s), a refinement is started. If skip is defined as True
149                       (or any other value) the refinement step is not performed.
150output                 If a file name is specified for output is will be used to save
151                       the current refinement.
152histograms             Should contain a list of histogram(s) to be used for the
153                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
154                       :ref:`HAP_parameters_table`. Note that this will be
155                       ignored for :ref:`Phase_parameters_table`. Histograms may be
156                       specified as a list of strings [('PWDR ...'),...], indices
157                       [0,1,2] or as list of objects [hist1, hist2].
158phases                 Should contain a list of phase(s) to be used for the
159                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
160                       :ref:`HAP_parameters_table`. Note that this will be
161                       ignored for :ref:`Histogram_parameters_table`.
162                       Phases may be specified as a list of strings
163                       [('Phase name'),...], indices [0,1,2] or as list of objects
164                       [phase0, phase2].
165call                   Specifies a function to call after a refinement is completed.
166                       The value supplied can be the object (typically a function)
167                       that will be called or a string that will evaluate (in the
168                       namespace inside :meth:`G2Project.iter_refinements` where
169                       ``self`` references the project.)
170                       Nothing is called if this is not specified.
171callargs               Provides a list of arguments that will be passed to the function
172                       in call (if any). If call is defined and callargs is not, the
173                       current <tt>G2Project</tt> is passed as a single argument.
174========== ============================================================================
175
176An example that performs a series of refinement steps follows:
177
178.. code-block::  python
179
180    reflist = [
181            {"set": { "Limits": { "low": 0.7 },
182                      "Background": { "no. coeffs": 3,
183                                      "refine": True }}},
184            {"set": { "LeBail": True,
185                      "Cell": True }},
186            {"set": { "Sample Parameters": ["DisplaceX"]}},
187            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
188            {"set": { "Mustrain": { "type": "uniaxial",
189                                    "refine": "equatorial",
190                                    "direction": [0, 0, 1]}}},
191            {"set": { "Mustrain": { "type": "uniaxial",
192                                    "refine": "axial"}}},
193            {"clear": { "LeBail": True},
194             "set": { "Atoms": { "Mn": "X" }}},
195            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
196    my_project.do_refinements(reflist)
197   
198
199In this example, a separate refinement step will be performed for each dict in the list (since
200"skip" is not included).
201Note that in the second from last refinement step, parameters are both set and cleared.
202   
203.. _Refinement_parameters_kinds:
204
205----------------------------
206Refinement parameter types
207----------------------------
208
209Note that parameters and refinement flags used in GSAS-II fall into three classes:
210
211    * **Histogram**: There will be a set of these for each dataset loaded into a
212      project file. The parameters available depend on the type of histogram
213      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
214      include the overall scale factor, background, instrument and sample parameters;
215      see the :ref:`Histogram_parameters_table` table for a list of the histogram
216      parameters where access has been provided.
217     
218    * **Phase**: There will be a set of these for each phase loaded into a
219      project file. While some parameters are found in all types of phases,
220      others are only found in certain types (modulated, magnetic, protein...).
221      Typical phase parameters include unit cell lengths and atomic positions; see the
222      :ref:`Phase_parameters_table` table for a list of the phase     
223      parameters where access has been provided.
224     
225    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
226      that is associated with each phase, so that if there are ``N`` phases and ``M``
227      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
228      histograms are not linked to all phases.) Typical HAP parameters include the
229      phase fractions, sample microstrain and crystallite size broadening terms,
230      hydrostatic strain perturbations of the unit cell and preferred orientation
231      values.
232      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
233      been provided.
234
235.. _Refinement_parameters_fmt:
236
237=================================
238Specifying Refinement Parameters
239=================================
240
241Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
242where the details of these dicts are organized depends on the
243type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
244of keys (as described below) for each of the three types of parameters.
245
246.. _Histogram_parameters_table:
247
248--------------------
249Histogram parameters
250--------------------
251
252This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
253and :func:`G2PwdrData.clear_refinements`. As an example,
254
255.. code-block::  python
256
257   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
258                         "Sample Parameters": ["Scale"],
259                         "Limits": [10000, 40000]})
260
261With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
262``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
263key is used to define specific histograms. As an example:
264
265.. code-block::  python
266
267  gsas_proj.do_refinements([
268      {'set': {
269          'Background': {'no.coeffs': 3, 'refine': True},
270          'Sample Parameters': ['Scale'],
271          'Limits': [10000, 40000]},
272      'histograms': [1,2]}
273                            ])
274
275Note that below in the Instrument Parameters section,
276related profile parameters (such as U and V) are grouped together but
277separated by commas to save space in the table.
278
279.. tabularcolumns:: |l|l|p{3.5in}|
280
281===================== ====================  =================================================
282key                   subkey                explanation
283===================== ====================  =================================================
284Limits                                      The range of 2-theta (degrees) or TOF (in
285                                            microsec) range of values to use. Can
286                                            be either a dictionary of 'low' and/or 'high',
287                                            or a list of 2 items [low, high]
288\                     low                   Sets the low limit
289\                     high                  Sets the high limit
290
291Sample Parameters                           Should be provided as a **list** of subkeys
292                                            to set or clear, e.g. ['DisplaceX', 'Scale']
293\                     Absorption
294\                     Contrast
295\                     DisplaceX             Sample displacement along the X direction
296\                     DisplaceY             Sample displacement along the Y direction
297\                     Scale                 Histogram Scale factor
298
299Background                                  Sample background. If value is a boolean,
300                                            the background's 'refine' parameter is set
301                                            to the given boolean. Usually should be a
302                                            dictionary with any of the following keys:
303\                     type                  The background model, e.g. 'chebyschev'
304\                     refine                The value of the refine flag, boolean
305\                     no. coeffs            Number of coefficients to use, integer
306\                     coeffs                List of floats, literal values for background
307\                     FixedPoints           List of (2-theta, intensity) values for fixed points
308\                     fit fixed points      If True, triggers a fit to the fixed points to
309                                            be calculated. It is calculated when this key is
310                                            detected, regardless of calls to refine.
311
312Instrument Parameters                       As in Sample Paramters, provide as a **list** of
313                                            subkeys to
314                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
315\                     U, V, W               Gaussian peak profile terms
316\                     X, Y, Z               Lorentzian peak profile terms
317\                     alpha, beta-0,        TOF profile terms
318                      beta-1, beta-q,
319\                     sig-0, sig-1,         TOF profile terms
320                      sig-2, sig-q
321\                     difA, difB, difC      TOF Calibration constants
322\                     Zero                  Zero shift
323\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
324\                     Polariz.              Polarization parameter
325\                     Lam                   Lambda, the incident wavelength
326===================== ====================  =================================================
327
328.. _Phase_parameters_table:
329
330----------------
331Phase parameters
332----------------
333
334This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
335and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
336these parameters should be placed inside a dict with a key
337``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
338key is used to define specific phase(s).
339
340
341.. tabularcolumns:: |l|p{4.5in}|
342
343======= ==========================================================
344key                   explanation
345======= ==========================================================
346Cell                  Whether or not to refine the unit cell.
347Atoms                 Dictionary of atoms and refinement flags.
348                      Each key should be an atom label, e.g.
349                      'O3', 'Mn5', and each value should be
350                      a string defining what values to refine.
351                      Values can be any combination of 'F'
352                      for fractional occupancy, 'X' for position,
353                      and 'U' for Debye-Waller factor
354LeBail                Enables LeBail intensity extraction.
355======= ==========================================================
356
357
358.. _HAP_parameters_table:
359
360
361Histogram-and-phase parameters
362------------------------------
363
364This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
365and :func:`G2Phase.clear_HAP_refinements`. When supplied to
366:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
367``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
368unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
369
370.. tabularcolumns:: |l|l|p{3.5in}|
371
372=============  ==========  ============================================================
373key             subkey                 explanation
374=============  ==========  ============================================================
375Babinet                                Should be a **list** of the following
376                                       subkeys. If not, assumes both
377                                       BabA and BabU
378\               BabA
379\               BabU
380Extinction                             Boolean, True to refine.
381HStrain                                Boolean, True to refine all appropriate
382                                       $D_ij$ terms.
383Mustrain
384\               type                   Mustrain model. One of 'isotropic',
385                                       'uniaxial', or 'generalized'. Should always
386                                       be specified.
387\              direction               For uniaxial only. A list of three
388                                       integers,
389                                       the [hkl] direction of the axis.
390\               refine                 Usually boolean, set to True to refine.
391                                       or False to clear.
392                                       For uniaxial model, can specify a value
393                                       of 'axial' or 'equatorial' to set that flag
394                                       to True or a single
395                                       boolean sets both axial and equatorial.
396Size                                   
397\               type                   Size broadening model. One of 'isotropic',
398                                       'uniaxial', or 'ellipsoid'. Should always
399                                       be specified.
400\              direction               For uniaxial only. A list of three
401                                       integers,
402                                       the [hkl] direction of the axis.
403\               refine                 Boolean, True to refine.
404\               value                  float, size value in microns
405Pref.Ori.                              Boolean, True to refine
406Show                                   Boolean, True to refine
407Use                                    Boolean, True to refine
408Scale                                  Phase fraction; Boolean, True to refine
409=============  ==========  ============================================================
410
411------------------------
412Histogram/Phase objects
413------------------------
414Each phase and powder histogram in a :class:`G2Project` object has an associated
415object. Parameters within each individual object can be turned on and off by calling
416:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
417for histogram parameters;
418:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
419for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
420:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
421
422.. code-block::  python
423
424    params = { 'Limits': [0.8, 12.0],
425               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
426               'Background': {'type': 'chebyschev', 'refine': True}}
427    some_histogram.set_refinements(params)
428
429Likewise to turn refinement flags on, use code such as this:
430
431.. code-block::  python
432
433    params = { 'Instrument Parameters': ['U', 'V', 'W']}
434    some_histogram.set_refinements(params)
435
436and to turn these refinement flags, off use this (Note that the
437``.clear_refinements()`` methods will usually will turn off refinement even
438if a refinement parameter is set in the dict to True.):
439
440.. code-block::  python
441
442    params = { 'Instrument Parameters': ['U', 'V', 'W']}
443    some_histogram.clear_refinements(params)
444
445For phase parameters, use code such as this:
446   
447.. code-block::  python
448
449    params = { 'LeBail': True, 'Cell': True,
450               'Atoms': { 'Mn1': 'X',
451                          'O3': 'XU',
452                          'V4': 'FXU'}}
453    some_histogram.set_refinements(params)
454
455
456and here is an example for HAP parameters:
457
458.. code-block::  python
459
460    params = { 'Babinet': 'BabA',
461               'Extinction': True,
462               'Mustrain': { 'type': 'uniaxial',
463                             'direction': [0, 0, 1],
464                             'refine': True}}
465    some_phase.set_HAP_refinements(params)
466
467Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
468
469.. _CommandlineInterface:
470
471=======================================
472GSASIIscriptable Command-line Interface
473=======================================
474
475The routines described above are intended to be called from a Python script, but an
476alternate way to access some of the same functionality is to
477invoke the ``GSASIIscriptable.py`` script from
478the command line usually from within a shell script or batch file. This
479will usually be done with a command such as::
480
481       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
482
483    The following subcommands are defined:
484
485        * create, see :func:`create`
486        * add, see :func:`add`
487        * dump, see :func:`dump`
488        * refine, see :func:`refine`
489        * seqrefine, see :func:`seqrefine`
490        * export, :func:`export`
491        * browse, see :func:`IPyBrowse`
492       
493Run::
494
495   python GSASIIscriptable.py --help
496
497to show the available subcommands, and inspect each subcommand with
498`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
499
500.. _JsonFormat:
501
502-------------------------
503Parameters in JSON files
504-------------------------
505
506The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
507a JSON format file
508(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
509This dict may have two keys:
510
511refinements:
512  This defines the a set of refinement steps in a JSON representation of a
513  :ref:`Refinement_recipe` list.
514
515code:
516  This optionally defines Python code that will be executed after the project is loaded,
517  but before the refinement is started. This can be used to execute Python code to change
518  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
519  project object is accessed with variable ``proj``) or to define code that will be called
520  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
521   
522JSON website: `Introducing JSON <http://json.org/>`_.
523
524.. _API:
525
526============================================================
527GSASIIscriptable Application Layer (API)
528============================================================
529
530The large number of classes and modules in this module are described below.
531Most commonly a script will create a G2Project object using :class:`G2Project` and then
532perform actions such as
533adding a histogram (method :meth:`G2Project.add_powder_histogram`),
534adding a phase (method :meth:`G2Project.add_phase`),
535or setting parameters and performing a refinement
536(method :meth:`G2Project.do_refinements`).
537
538In some cases, it may be easier to use
539methods inside :class:`G2PwdrData` or :class:`G2Phase` or these objects
540may offer more options.
541
542---------------------------------------------------------------
543Complete Documentation: All classes and functions
544---------------------------------------------------------------
545"""
546from __future__ import division, print_function
547import argparse
548import os.path as ospath
549import datetime as dt
550import sys
551import platform
552if '2' in platform.python_version_tuple()[0]:
553    import cPickle
554    strtypes = (str,unicode)
555else:
556    import _pickle as cPickle
557    strtypes = (str,bytes)
558import imp
559import copy
560import os
561import random as ran
562
563import numpy.ma as ma
564import scipy.interpolate as si
565import numpy as np
566import scipy as sp
567
568import GSASIIpath
569GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
570import GSASIIobj as G2obj
571import GSASIIpwd as G2pwd
572import GSASIIstrMain as G2strMain
573import GSASIIspc as G2spc
574import GSASIIElem as G2elem
575
576
577# Delay imports to not slow down small scripts
578G2fil = None
579PwdrDataReaders = []
580PhaseReaders = []
581exportersByExtension = {}
582'''Specifies the list of extensions that are supported for Powder data export'''
583
584def LoadG2fil():
585    """Delay importing this module, it is slow"""
586    global G2fil
587    if G2fil is None:
588        import GSASIIfiles
589        G2fil = GSASIIfiles
590        global PwdrDataReaders
591        global PhaseReaders
592        PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
593        PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
594        AllExporters = G2fil.LoadExportRoutines(None)
595        global exportersByExtension
596        exportersByExtension = {}
597        for obj in AllExporters:
598            try:
599                obj.Writer
600            except AttributeError:
601                continue
602            for typ in obj.exporttype:
603                if typ not in exportersByExtension:
604                    exportersByExtension[typ] = {obj.extension:obj}
605                else:
606                    exportersByExtension[typ][obj.extension] = obj
607
608def LoadDictFromProjFile(ProjFile):
609    '''Read a GSAS-II project file and load items to dictionary
610   
611    :param str ProjFile: GSAS-II project (name.gpx) full file name
612    :returns: Project,nameList, where
613
614      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
615        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
616        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
617      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
618
619    Example for fap.gpx::
620
621      Project = {                 #NB:dict order is not tree order
622        u'Phases':{'data':None,'fap':{phase dict}},
623        u'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
624        u'Rigid bodies':{'data': {rigid body dict}},
625        u'Covariance':{'data':{covariance data dict}},
626        u'Controls':{'data':{controls data dict}},
627        u'Notebook':{'data':[notebook list]},
628        u'Restraints':{'data':{restraint data dict}},
629        u'Constraints':{'data':{constraint data dict}}]
630        }
631      nameList = [                #NB: reproduces tree order
632        [u'Notebook',],
633        [u'Controls',],
634        [u'Covariance',],
635        [u'Constraints',],
636        [u'Restraints',],
637        [u'Rigid bodies',],
638        [u'PWDR FAP.XRA Bank 1',
639             u'Comments',
640             u'Limits',
641             u'Background',
642             u'Instrument Parameters',
643             u'Sample Parameters',
644             u'Peak List',
645             u'Index Peak List',
646             u'Unit Cells List',
647             u'Reflection Lists'],
648        [u'Phases', u'fap']
649        ]
650    '''
651    # Let IOError be thrown if file does not exist
652    # if not ospath.exists(ProjFile):
653    #     print ('\n*** Error attempt to open project file that does not exist:\n   '+
654    #         str(ProjFile))
655    #     return
656    file = open(ProjFile,'rb')
657    # print('loading from file: {}'.format(ProjFile))
658    Project = {}
659    nameList = []
660    try:
661        while True:
662            try:
663                data = cPickle.load(file)
664            except EOFError:
665                break
666            datum = data[0]
667            Project[datum[0]] = {'data':datum[1]}
668            nameList.append([datum[0],])
669            for datus in data[1:]:
670                Project[datum[0]][datus[0]] = datus[1]
671                nameList[-1].append(datus[0])
672        file.close()
673        # print('project load successful')
674    except:
675        raise IOError("Error reading file "+str(ProjFile)+". This is not a GSAS-II .gpx file")
676    finally:
677        file.close()
678    return Project,nameList
679
680def SaveDictToProjFile(Project,nameList,ProjFile):
681    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
682
683    :param dict Project: representation of gpx file following the GSAS-II
684        tree structure as described for LoadDictFromProjFile
685    :param list nameList: names of main tree entries & subentries used to reconstruct project file
686    :param str ProjFile: full file name for output project.gpx file (including extension)
687    '''
688    file = open(ProjFile,'wb')
689    try:
690        for name in nameList:
691            data = []
692            item = Project[name[0]]
693            data.append([name[0],item['data']])
694            for item2 in name[1:]:
695                data.append([item2,item[item2]])
696            cPickle.dump(data,file,1)
697    finally:
698        file.close()
699
700# def ImportPowder(reader,filename):
701#     '''Use a reader to import a powder diffraction data file
702
703#     :param str reader: a scriptable reader
704#     :param str filename: full name of powder data file; can be "multi-Bank" data
705
706#     :returns: list rdlist: list of reader objects containing powder data, one for each
707#         "Bank" of data encountered in file. Items in reader object of interest are:
708
709#           * rd.comments: list of str: comments found on powder file
710#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
711#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
712#     '''
713#     rdfile,rdpath,descr = imp.find_module(reader)
714#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
715#     rd = rdclass.GSAS_ReaderClass()
716#     if not rd.scriptable:
717#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
718#         return None
719#     rdlist = []
720#     if rd.ContentsValidator(filename):
721#         repeat = True
722#         rdbuffer = {} # create temporary storage for file reader
723#         block = 0
724#         while repeat: # loop if the reader asks for another pass on the file
725#             block += 1
726#             repeat = False
727#             rd.objname = ospath.basename(filename)
728#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
729#             if flag:
730#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
731#                 if rd.repeat:
732#                     repeat = True
733#         return rdlist
734#     print(rd.errors)
735#     return None
736
737def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
738    '''Create an initial Histogram dictionary
739
740    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
741    '''
742    if dType in ['SXC','SNC']:
743        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
744            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
745            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
746            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
747            'Flack':[0.0,False]}
748    elif dType == 'SNT':
749        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
750            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
751            'Extinction':['Lorentzian','None', {
752            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
753    elif 'P' in dType:
754        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
755            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
756            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
757                [1.,1.,1.,0.,0.,0.],6*[False,]],
758            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
759                NShkl*[0.01,],NShkl*[False,]],
760            'HStrain':[NDij*[0.0,],NDij*[False,]],
761            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
762
763
764def PreSetup(data):
765    '''Create part of an initial (empty) phase dictionary
766
767    from GSASIIphsGUI.py, near end of UpdatePhaseData
768
769    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
770    '''
771    if 'RBModels' not in data:
772        data['RBModels'] = {}
773    if 'MCSA' not in data:
774        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
775    if 'dict' in str(type(data['MCSA']['Results'])):
776        data['MCSA']['Results'] = []
777    if 'Modulated' not in data['General']:
778        data['General']['Modulated'] = False
779#    if 'modulated' in data['General']['Type']:
780#        data['General']['Modulated'] = True
781#        data['General']['Type'] = 'nuclear'
782
783
784def SetupGeneral(data, dirname):
785    """Helps initialize phase data.
786
787    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
788
789    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
790    """
791    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
792                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
793    generalData = data['General']
794    atomData = data['Atoms']
795    generalData['AtomTypes'] = []
796    generalData['Isotopes'] = {}
797
798    if 'Isotope' not in generalData:
799        generalData['Isotope'] = {}
800    if 'Data plot type' not in generalData:
801        generalData['Data plot type'] = 'Mustrain'
802    if 'POhkl' not in generalData:
803        generalData['POhkl'] = [0,0,1]
804    if 'Map' not in generalData:
805        generalData['Map'] = mapDefault.copy()
806    if 'Flip' not in generalData:
807        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
808            'k-factor':0.1,'k-Max':20.,}
809    if 'testHKL' not in generalData['Flip']:
810        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
811    if 'doPawley' not in generalData:
812        generalData['doPawley'] = False     #ToDo: change to ''
813    if 'Pawley dmin' not in generalData:
814        generalData['Pawley dmin'] = 1.0
815    if 'Pawley neg wt' not in generalData:
816        generalData['Pawley neg wt'] = 0.0
817    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
818        'MCSA controls' not in generalData:
819        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
820        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
821        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
822    if 'AtomPtrs' not in generalData:
823        generalData['AtomPtrs'] = [3,1,7,9]
824        if generalData['Type'] == 'macromolecular':
825            generalData['AtomPtrs'] = [6,4,10,12]
826        elif generalData['Type'] == 'magnetic':
827            generalData['AtomPtrs'] = [3,1,10,12]
828    if generalData['Modulated']:
829        generalData['Type'] = 'nuclear'
830        if 'Super' not in generalData:
831            generalData['Super'] = 1
832            generalData['SuperVec'] = [[0,0,.1],False,4]
833            generalData['SSGData'] = {}
834        if '4DmapData' not in generalData:
835            generalData['4DmapData'] = mapDefault.copy()
836            generalData['4DmapData'].update({'MapType':'Fobs'})
837    if 'Modulated' not in generalData:
838        generalData['Modulated'] = False
839    if 'HydIds' not in generalData:
840        generalData['HydIds'] = {}
841    cx,ct,cs,cia = generalData['AtomPtrs']
842    generalData['NoAtoms'] = {}
843    generalData['BondRadii'] = []
844    generalData['AngleRadii'] = []
845    generalData['vdWRadii'] = []
846    generalData['AtomMass'] = []
847    generalData['Color'] = []
848    if generalData['Type'] == 'magnetic':
849        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
850        landeg = generalData.get('Lande g',[])
851    generalData['Mydir'] = dirname
852    badList = {}
853    for iat,atom in enumerate(atomData):
854        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
855        if generalData['AtomTypes'].count(atom[ct]):
856            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
857        elif atom[ct] != 'UNK':
858            Info = G2elem.GetAtomInfo(atom[ct])
859            if not Info:
860                if atom[ct] not in badList:
861                    badList[atom[ct]] = 0
862                badList[atom[ct]] += 1
863                atom[ct] = 'UNK'
864                continue
865            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
866            generalData['AtomTypes'].append(atom[ct])
867            generalData['Z'] = Info['Z']
868            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
869            generalData['BondRadii'].append(Info['Drad'])
870            generalData['AngleRadii'].append(Info['Arad'])
871            generalData['vdWRadii'].append(Info['Vdrad'])
872            if atom[ct] in generalData['Isotope']:
873                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
874                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
875                    generalData['Isotope'][atom[ct]] = isotope
876                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
877            else:
878                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
879                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
880                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
881                    generalData['Isotope'][atom[ct]] = isotope
882                generalData['AtomMass'].append(Info['Mass'])
883            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
884            generalData['Color'].append(Info['Color'])
885            if generalData['Type'] == 'magnetic':
886                if len(landeg) < len(generalData['AtomTypes']):
887                    landeg.append(2.0)
888    if generalData['Type'] == 'magnetic':
889        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
890
891    if badList:
892        msg = 'Warning: element symbol(s) not found:'
893        for key in badList:
894            msg += '\n\t' + key
895            if badList[key] > 1:
896                msg += ' (' + str(badList[key]) + ' times)'
897        raise G2ScriptException("Phase error:\n" + msg)
898        # wx.MessageBox(msg,caption='Element symbol error')
899    F000X = 0.
900    F000N = 0.
901    for i,elem in enumerate(generalData['AtomTypes']):
902        F000X += generalData['NoAtoms'][elem]*generalData['Z']
903        isotope = generalData['Isotope'][elem]
904        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
905    generalData['F000X'] = F000X
906    generalData['F000N'] = F000N
907    import GSASIImath as G2mth
908    generalData['Mass'] = G2mth.getMass(generalData)
909
910
911def make_empty_project(author=None, filename=None):
912    """Creates an dictionary in the style of GSASIIscriptable, for an empty
913    project.
914
915    If no author name or filename is supplied, 'no name' and
916    <current dir>/test_output.gpx are used , respectively.
917
918    Returns: project dictionary, name list
919
920    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
921    """
922    if not filename:
923        filename = 'test_output.gpx'
924    filename = os.path.abspath(filename)
925    gsasii_version = str(GSASIIpath.GetVersionNumber())
926    LoadG2fil()
927    import matplotlib as mpl
928    python_library_versions = G2fil.get_python_versions([mpl, np, sp])
929
930    controls_data = dict(G2obj.DefaultControls)
931    controls_data['LastSavedAs'] = filename
932    controls_data['LastSavedUsing'] = gsasii_version
933    controls_data['PythonVersions'] = python_library_versions
934    if author:
935        controls_data['Author'] = author
936
937    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
938                                       'Global': []}},
939              'Controls': {'data': controls_data},
940              u'Covariance': {'data': {}},
941              u'Notebook': {'data': ['']},
942              u'Restraints': {'data': {}},
943              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
944                                'Residue': {'AtInfo': {}},
945                                'Vector':  {'AtInfo': {}}}}}
946
947    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
948             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
949
950    return output, names
951
952
953class G2ImportException(Exception):
954    pass
955
956class G2ScriptException(Exception):
957    pass
958
959def import_generic(filename, readerlist, fmthint=None):
960    """Attempt to import a filename, using a list of reader objects.
961
962    Returns the first reader object which worked."""
963    # Translated from OnImportGeneric method in GSASII.py
964    primaryReaders, secondaryReaders = [], []
965    for reader in readerlist:
966        if fmthint is not None and fmthint not in reader.formatName: continue
967        flag = reader.ExtensionValidator(filename)
968        if flag is None:
969            secondaryReaders.append(reader)
970        elif flag:
971            primaryReaders.append(reader)
972    if not secondaryReaders and not primaryReaders:
973        raise G2ImportException("Could not read file: ", filename)
974
975    with open(filename, 'Ur') as fp:
976        rd_list = []
977
978        for rd in primaryReaders + secondaryReaders:
979            # Initialize reader
980            rd.selections = []
981            rd.dnames = []
982            rd.ReInitialize()
983            # Rewind file
984            rd.errors = ""
985            if not rd.ContentsValidator(filename):
986                # Report error
987                pass
988            if len(rd.selections) > 1:
989                # Select data?
990                # GSASII.py:543
991                raise G2ImportException("Not sure what data to select")
992
993            block = 0
994            rdbuffer = {}
995            repeat = True
996            while repeat:
997                repeat = False
998                block += 1
999                rd.objname = os.path.basename(filename)
1000                try:
1001                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
1002                except:
1003                    flag = False
1004                if flag:
1005                    # Omitting image loading special cases
1006                    rd.readfilename = filename
1007                    rd_list.append(copy.deepcopy(rd))
1008                    repeat = rd.repeat
1009                else:
1010                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
1011            if rd_list:
1012                if rd.warnings:
1013                    print("Read warning by", rd.formatName, "reader:",
1014                          rd.warnings, file=sys.stderr)
1015                else:
1016                    print("{} read by Reader {}\n".format(filename,rd.formatName))                   
1017                return rd_list
1018    raise G2ImportException("No reader could read file: " + filename)
1019
1020
1021def load_iprms(instfile, reader):
1022    """Loads instrument parameters from a file, and edits the
1023    given reader.
1024
1025    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1026    """
1027    LoadG2fil()
1028    ext = os.path.splitext(instfile)[1]
1029
1030    if ext.lower() == '.instprm':
1031        # New GSAS File, load appropriately
1032        with open(instfile) as f:
1033            lines = f.readlines()
1034        bank = reader.powderentry[2]
1035        numbanks = reader.numbanks
1036        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1037
1038        reader.instfile = instfile
1039        reader.instmsg = 'GSAS-II file' + instfile
1040        return iparms
1041    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1042        raise ValueError('Expected .prm file, found: ', instfile)
1043
1044    # It's an old GSAS file, load appropriately
1045    Iparm = {}
1046    with open(instfile, 'Ur') as fp:
1047        for line in fp:
1048            if '#' in line:
1049                continue
1050            Iparm[line[:12]] = line[12:-1]
1051    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1052    if ibanks == 1:
1053        reader.instbank = 1
1054        reader.powderentry[2] = 1
1055        reader.instfile = instfile
1056        reader.instmsg = instfile + ' bank ' + str(reader.instbank)
1057        return G2fil.SetPowderInstParms(Iparm, reader)
1058    # TODO handle >1 banks
1059    raise NotImplementedError("Check GSASIIfiles.py: ReadPowderInstprm")
1060
1061def load_pwd_from_reader(reader, instprm, existingnames=[]):
1062    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1063
1064    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1065
1066    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1067    """
1068    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1069    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1070
1071    try:
1072        Iparm1, Iparm2 = instprm
1073    except ValueError:
1074        Iparm1, Iparm2 = load_iprms(instprm, reader)
1075
1076    Ymin = np.min(reader.powderdata[1])
1077    Ymax = np.max(reader.powderdata[1])
1078    valuesdict = {'wtFactor': 1.0,
1079                  'Dummy': False,
1080                  'ranId': ran.randint(0, sys.maxsize),
1081                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1082                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1083                  'Yminmax': [Ymin, Ymax]}
1084    reader.Sample['ranId'] = valuesdict['ranId']
1085
1086    # Ending keys:
1087    # [u'Reflection Lists',
1088    #  u'Limits',
1089    #  'data',
1090    #  u'Index Peak List',
1091    #  u'Comments',
1092    #  u'Unit Cells List',
1093    #  u'Sample Parameters',
1094    #  u'Peak List',
1095    #  u'Background',
1096    #  u'Instrument Parameters']
1097    Tmin = np.min(reader.powderdata[0])
1098    Tmax = np.max(reader.powderdata[0])
1099
1100    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1101                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1102
1103    output_dict = {u'Reflection Lists': {},
1104                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1105                   u'data': [valuesdict, reader.powderdata, HistName],
1106                   u'Index Peak List': [[], []],
1107                   u'Comments': reader.comments,
1108                   u'Unit Cells List': [],
1109                   u'Sample Parameters': reader.Sample,
1110                   u'Peak List': {'peaks': [], 'sigDict': {}},
1111                   u'Background': reader.pwdparms.get('Background', default_background),
1112                   u'Instrument Parameters': [Iparm1, Iparm2],
1113                   }
1114
1115    names = [u'Comments',
1116             u'Limits',
1117             u'Background',
1118             u'Instrument Parameters',
1119             u'Sample Parameters',
1120             u'Peak List',
1121             u'Index Peak List',
1122             u'Unit Cells List',
1123             u'Reflection Lists']
1124
1125    # TODO controls?? GSASII.py:1664-7
1126
1127    return HistName, [HistName] + names, output_dict
1128
1129
1130def _deep_copy_into(from_, into):
1131    """Helper function for reloading .gpx file. See G2Project.reload()
1132
1133    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1134    """
1135    if isinstance(from_, dict) and isinstance(into, dict):
1136        combined_keys = set(from_.keys()).union(into.keys())
1137        for key in combined_keys:
1138            if key in from_ and key in into:
1139                both_dicts = (isinstance(from_[key], dict)
1140                              and isinstance(into[key], dict))
1141                both_lists = (isinstance(from_[key], list)
1142                              and isinstance(into[key], list))
1143                if both_dicts or both_lists:
1144                    _deep_copy_into(from_[key], into[key])
1145                else:
1146                    into[key] = from_[key]
1147            elif key in from_:
1148                into[key] = from_[key]
1149            else:  # key in into
1150                del into[key]
1151    elif isinstance(from_, list) and isinstance(into, list):
1152        if len(from_) == len(into):
1153            for i in range(len(from_)):
1154                both_dicts = (isinstance(from_[i], dict)
1155                              and isinstance(into[i], dict))
1156                both_lists = (isinstance(from_[i], list)
1157                              and isinstance(into[i], list))
1158                if both_dicts or both_lists:
1159                    _deep_copy_into(from_[i], into[i])
1160                else:
1161                    into[i] = from_[i]
1162        else:
1163            into[:] = from_
1164
1165
1166class G2ObjectWrapper(object):
1167    """Base class for all GSAS-II object wrappers.
1168
1169    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1170    of overrides are implemented so that the wrapper behaves like a dictionary.
1171
1172    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1173    """
1174    def __init__(self, datadict):
1175        self.data = datadict
1176
1177    def __getitem__(self, key):
1178        return self.data[key]
1179
1180    def __setitem__(self, key, value):
1181        self.data[key] = value
1182
1183    def __contains__(self, key):
1184        return key in self.data
1185
1186    def get(self, k, d=None):
1187        return self.data.get(k, d)
1188
1189    def keys(self):
1190        return self.data.keys()
1191
1192    def values(self):
1193        return self.data.values()
1194
1195    def items(self):
1196        return self.data.items()
1197
1198
1199class G2Project(G2ObjectWrapper):
1200    """
1201    Represents an entire GSAS-II project.
1202
1203    There are two ways to initialize it:
1204
1205    >>> # Load an existing project file
1206    >>> proj = G2Project('filename.gpx')
1207   
1208    >>> # Create a new project
1209    >>> proj = G2Project(newgpx='new_file.gpx')
1210   
1211    Histograms can be accessed easily.
1212
1213    >>> # By name
1214    >>> hist = proj.histogram('PWDR my-histogram-name')
1215   
1216    >>> # Or by index
1217    >>> hist = proj.histogram(0)
1218    >>> assert hist.id == 0
1219   
1220    >>> # Or by random id
1221    >>> assert hist == proj.histogram(hist.ranId)
1222
1223    Phases can be accessed the same way.
1224
1225    >>> phase = proj.phase('name of phase')
1226
1227    New data can also be loaded via :meth:`~G2Project.add_phase` and
1228    :meth:`~G2Project.add_powder_histogram`.
1229
1230    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1231                                         'instrument_parameters.prm')
1232    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1233
1234    Parameters for Rietveld refinement can be turned on and off as well.
1235    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1236    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1237    """
1238    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1239        """Loads a GSAS-II project from a specified filename.
1240
1241        :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1242            creates an empty project.
1243        :param str author: Author's name (not yet implemented)
1244        :param str newgpx: The filename the project should be saved to in
1245            the future. If both newgpx and gpxfile are present, the project is
1246            loaded from the gpxfile, then when saved will be written to newgpx.
1247        :param str filename: Name to be used to save the project. Has same function as
1248            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1249            is preferred over filename.
1250        """
1251        if filename is not None and newgpx is not None:
1252            raise G2ScriptException('Do not use filename and newgpx together')
1253        elif newgpx is not None:
1254            filename = newgpx
1255        if gpxfile is None:
1256            filename = os.path.abspath(os.path.expanduser(filename))
1257            self.filename = filename
1258            self.data, self.names = make_empty_project(author=author, filename=filename)
1259        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1260                                       # filename is valid, etc.
1261            if isinstance(filename, str): 
1262                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1263            else: 
1264                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1265            # TODO set author
1266            self.data, self.names = LoadDictFromProjFile(gpxfile)
1267            self.update_ids()
1268        else:
1269            raise ValueError("Not sure what to do with gpxfile")
1270
1271    @classmethod
1272    def from_dict_and_names(cls, gpxdict, names, filename=None):
1273        """Creates a :class:`G2Project` directly from
1274        a dictionary and a list of names. If in doubt, do not use this.
1275
1276        :returns: a :class:`G2Project`
1277        """
1278        out = cls()
1279        if filename:
1280            filename = os.path.abspath(os.path.expanduser(filename))
1281            out.filename = filename
1282            gpxdict['Controls']['data']['LastSavedAs'] = filename
1283        else:
1284            try:
1285                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1286            except KeyError:
1287                out.filename = None
1288        out.data = gpxdict
1289        out.names = names
1290
1291    def save(self, filename=None):
1292        """Saves the project, either to the current filename, or to a new file.
1293
1294        Updates self.filename if a new filename provided"""
1295        # TODO update LastSavedUsing ?
1296        if filename:
1297            filename = os.path.abspath(os.path.expanduser(filename))
1298            self.data['Controls']['data']['LastSavedAs'] = filename
1299            self.filename = filename
1300        elif not self.filename:
1301            raise AttributeError("No file name to save to")
1302        SaveDictToProjFile(self.data, self.names, self.filename)
1303
1304    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None):
1305        """Loads a powder data histogram into the project.
1306
1307        Automatically checks for an instrument parameter file, or one can be
1308        provided. Note that in unix fashion, "~" can be used to indicate the
1309        home directory (e.g. ~/G2data/data.fxye).
1310
1311        :param str datafile: The powder data file to read, a filename.
1312        :param str iparams: The instrument parameters file, a filename.
1313        :param list phases: Phases to link to the new histogram
1314        :param str fmthint: If specified, only importers where the format name
1315          (reader.formatName, as shown in Import menu) contains the
1316          supplied string will be tried as importers. If not specified, all
1317          importers consistent with the file extension will be tried
1318          (equivalent to "guess format" in menu).
1319
1320        :returns: A :class:`G2PwdrData` object representing
1321            the histogram
1322        """
1323        LoadG2fil()
1324        datafile = os.path.abspath(os.path.expanduser(datafile))
1325        iparams = os.path.abspath(os.path.expanduser(iparams))
1326        pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint)
1327        histname, new_names, pwdrdata = load_pwd_from_reader(
1328                                          pwdrreaders[0], iparams,
1329                                          [h.name for h in self.histograms()])
1330        if histname in self.data:
1331            print("Warning - redefining histogram", histname)
1332        elif self.names[-1][0] == 'Phases':
1333            self.names.insert(-1, new_names)
1334        else:
1335            self.names.append(new_names)
1336        self.data[histname] = pwdrdata
1337        self.update_ids()
1338
1339        for phase in phases:
1340            phase = self.phase(phase)
1341            self.link_histogram_phase(histname, phase)
1342
1343        return self.histogram(histname)
1344
1345    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1346                                       wavelength=None, scale=None, phases=[]):
1347        """Loads a powder data histogram into the project.
1348
1349        Requires an instrument parameter file.
1350        Note that in unix fashion, "~" can be used to indicate the
1351        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1352        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1353        as the instrument type.
1354
1355        :param str histname: A name for the histogram to be created.
1356        :param str iparams: The instrument parameters file, a filename.
1357        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1358        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1359        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1360        :param float wavelength: Wavelength for CW instruments, overriding the value
1361           in the instrument parameters file if specified.
1362        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1363           simulated noise is added to the pattern, so that if the maximum intensity is
1364           small, the noise will mask the computed pattern. The scale
1365           needs to be a large number for CW neutrons.
1366           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1367        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1368           all defined phases.
1369
1370        :returns: A :class:`G2PwdrData` object representing the histogram
1371        """
1372        LoadG2fil()
1373        iparams = os.path.abspath(os.path.expanduser(iparams))
1374        if not os.path.exists(iparams):
1375            raise G2ScriptException("File does not exist:"+iparams)
1376        rd = G2obj.ImportPowderData( # Initialize a base class reader
1377            extensionlist=tuple(),
1378            strictExtension=False,
1379            formatName = 'Simulate dataset',
1380            longFormatName = 'Compute a simulated pattern')
1381        rd.powderentry[0] = '' # no filename
1382        rd.powderentry[2] = 1 # only one bank
1383        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1384        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1385        if Tmax < Tmin:
1386            Tmin,Tmax = Tmax,Tmin
1387        Tstep = abs(Tstep)
1388        if 'TOF' in rd.idstring:
1389                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1390                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1391                N = len(x)
1392        else:           
1393                N = int((Tmax-Tmin)/Tstep)+1
1394                x = np.linspace(Tmin,Tmax,N,True)
1395                N = len(x)
1396        if N < 3:
1397            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1398        rd.powderdata = [
1399            np.array(x), # x-axis values
1400            np.zeros_like(x), # powder pattern intensities
1401            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1402            np.zeros_like(x), # calc. intensities (zero)
1403            np.zeros_like(x), # calc. background (zero)
1404            np.zeros_like(x), # obs-calc profiles
1405            ]
1406        Tmin = rd.powderdata[0][0]
1407        Tmax = rd.powderdata[0][-1]
1408        rd.idstring = histname
1409        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1410                                                            [h.name for h in self.histograms()])
1411        if histname in self.data:
1412            print("Warning - redefining histogram", histname)
1413        elif self.names[-1][0] == 'Phases':
1414            self.names.insert(-1, new_names)
1415        else:
1416            self.names.append(new_names)
1417        if scale is not None:
1418            pwdrdata['Sample Parameters']['Scale'][0] = scale
1419        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1420            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1421        self.data[histname] = pwdrdata
1422        self.update_ids()
1423
1424        for phase in phases:
1425            phase = self.phase(phase)
1426            self.link_histogram_phase(histname, phase)
1427
1428        return self.histogram(histname)
1429   
1430    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1431        """Loads a phase into the project from a .cif file
1432
1433        :param str phasefile: The CIF file from which to import the phase.
1434        :param str phasename: The name of the new phase, or None for the default
1435        :param list histograms: The names of the histograms to associate with
1436            this phase. Use proj.Histograms() to add to all histograms.
1437        :param str fmthint: If specified, only importers where the format name
1438          (reader.formatName, as shown in Import menu) contains the
1439          supplied string will be tried as importers. If not specified, all
1440          importers consistent with the file extension will be tried
1441          (equivalent to "guess format" in menu).
1442
1443        :returns: A :class:`G2Phase` object representing the
1444            new phase.
1445        """
1446        LoadG2fil()
1447        histograms = [self.histogram(h).name for h in histograms]
1448        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1449
1450        # TODO handle multiple phases in a file
1451        phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
1452        phasereader = phasereaders[0]
1453       
1454        phasename = phasename or phasereader.Phase['General']['Name']
1455        phaseNameList = [p.name for p in self.phases()]
1456        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1457        phasereader.Phase['General']['Name'] = phasename
1458
1459        if 'Phases' not in self.data:
1460            self.data[u'Phases'] = { 'data': None }
1461        assert phasename not in self.data['Phases'], "phase names should be unique"
1462        self.data['Phases'][phasename] = phasereader.Phase
1463
1464        if phasereader.Constraints:
1465            Constraints = self.data['Constraints']
1466            for i in phasereader.Constraints:
1467                if isinstance(i, dict):
1468                    if '_Explain' not in Constraints:
1469                        Constraints['_Explain'] = {}
1470                    Constraints['_Explain'].update(i)
1471                else:
1472                    Constraints['Phase'].append(i)
1473
1474        data = self.data['Phases'][phasename]
1475        generalData = data['General']
1476        SGData = generalData['SGData']
1477        NShkl = len(G2spc.MustrainNames(SGData))
1478        NDij = len(G2spc.HStrainNames(SGData))
1479        Super = generalData.get('Super', 0)
1480        if Super:
1481            SuperVec = np.array(generalData['SuperVec'][0])
1482        else:
1483            SuperVec = []
1484        UseList = data['Histograms']
1485
1486        for hist in histograms:
1487            self.link_histogram_phase(hist, phasename)
1488
1489        for obj in self.names:
1490            if obj[0] == 'Phases':
1491                phasenames = obj
1492                break
1493        else:
1494            phasenames = [u'Phases']
1495            self.names.append(phasenames)
1496        phasenames.append(phasename)
1497
1498        # TODO should it be self.filename, not phasefile?
1499        SetupGeneral(data, os.path.dirname(phasefile))
1500        self.index_ids()
1501
1502        self.update_ids()
1503        return self.phase(phasename)
1504
1505    def link_histogram_phase(self, histogram, phase):
1506        """Associates a given histogram and phase.
1507
1508        .. seealso::
1509
1510            :meth:`G2Project.histogram`
1511            :meth:`G2Project.phase`"""
1512        hist = self.histogram(histogram)
1513        phase = self.phase(phase)
1514
1515        generalData = phase['General']
1516
1517        if hist.name.startswith('HKLF '):
1518            raise NotImplementedError("HKLF not yet supported")
1519        elif hist.name.startswith('PWDR '):
1520            hist['Reflection Lists'][generalData['Name']] = {}
1521            UseList = phase['Histograms']
1522            SGData = generalData['SGData']
1523            NShkl = len(G2spc.MustrainNames(SGData))
1524            NDij = len(G2spc.HStrainNames(SGData))
1525            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1526            UseList[hist.name]['hId'] = hist.id
1527            for key, val in [('Use', True), ('LeBail', False),
1528                             ('newLeBail', True),
1529                             ('Babinet', {'BabA': [0.0, False],
1530                                          'BabU': [0.0, False]})]:
1531                if key not in UseList[hist.name]:
1532                    UseList[hist.name][key] = val
1533        else:
1534            raise RuntimeError("Unexpected histogram" + hist.name)
1535
1536
1537    def reload(self):
1538        """Reload self from self.filename"""
1539        data, names = LoadDictFromProjFile(self.filename)
1540        self.names = names
1541        # Need to deep copy the new data file data into the current tree,
1542        # so that any existing G2Phase, or G2PwdrData objects will still be
1543        # valid
1544        _deep_copy_into(from_=data, into=self.data)
1545
1546    def refine(self, newfile=None, printFile=None, makeBack=False):
1547        # TODO migrate to RefineCore
1548        # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1549        #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1550        # index_ids will automatically save the project
1551        self.index_ids()
1552        # TODO G2strMain does not properly use printFile
1553        G2strMain.Refine(self.filename, makeBack=makeBack)
1554        # Reload yourself
1555        self.reload()
1556
1557    def histogram(self, histname):
1558        """Returns the histogram named histname, or None if it does not exist.
1559
1560        :param histname: The name of the histogram (str), or ranId or index.
1561        :returns: A :class:`G2PwdrData` object, or None if
1562            the histogram does not exist
1563
1564        .. seealso::
1565            :meth:`G2Project.histograms`
1566            :meth:`G2Project.phase`
1567            :meth:`G2Project.phases`
1568            """
1569        if isinstance(histname, G2PwdrData):
1570            if histname.proj == self:
1571                return histname
1572        if histname in self.data:
1573            return G2PwdrData(self.data[histname], self)
1574        try:
1575            # see if histname is an id or ranId
1576            histname = int(histname)
1577        except ValueError:
1578            return
1579
1580        for histogram in self.histograms():
1581            if histogram.id == histname or histogram.ranId == histname:
1582                return histogram
1583
1584    def histograms(self):
1585        """Return a list of all histograms, as
1586        :class:`G2PwdrData` objects
1587
1588        .. seealso::
1589            :meth:`G2Project.histograms`
1590            :meth:`G2Project.phase`
1591            :meth:`G2Project.phases`
1592            """
1593        output = []
1594        for obj in self.names:
1595            if len(obj) > 1 and obj[0] != u'Phases':
1596                output.append(self.histogram(obj[0]))
1597        return output
1598
1599    def phase(self, phasename):
1600        """
1601        Gives an object representing the specified phase in this project.
1602
1603        :param str phasename: A reference to the desired phase. Either the phase
1604            name (str), the phase's ranId, the phase's index (both int) or
1605            a phase object (:class:`G2Phase`)
1606        :returns: A :class:`G2Phase` object
1607        :raises: KeyError
1608
1609        .. seealso::
1610            :meth:`G2Project.histograms`
1611            :meth:`G2Project.phase`
1612            :meth:`G2Project.phases`
1613            """
1614        if isinstance(phasename, G2Phase):
1615            if phasename.proj == self:
1616                return phasename
1617        phases = self.data['Phases']
1618        if phasename in phases:
1619            return G2Phase(phases[phasename], phasename, self)
1620
1621        try:
1622            # phasename should be phase index or ranId
1623            phasename = int(phasename)
1624        except ValueError:
1625            return
1626
1627        for phase in self.phases():
1628            if phase.id == phasename or phase.ranId == phasename:
1629                return phase
1630
1631    def phases(self):
1632        """
1633        Returns a list of all the phases in the project.
1634
1635        :returns: A list of :class:`G2Phase` objects
1636
1637        .. seealso::
1638            :meth:`G2Project.histogram`
1639            :meth:`G2Project.histograms`
1640            :meth:`G2Project.phase`
1641            """
1642        for obj in self.names:
1643            if obj[0] == 'Phases':
1644                return [self.phase(p) for p in obj[1:]]
1645        return []
1646
1647    def update_ids(self):
1648        """Makes sure all phases and histograms have proper hId and pId"""
1649        # Translated from GetUsedHistogramsAndPhasesfromTree,
1650        #   GSASIIdataGUI.py:4107
1651        for i, h in enumerate(self.histograms()):
1652            h.id = i
1653        for i, p in enumerate(self.phases()):
1654            p.id = i
1655
1656    def do_refinements(self, refinements, histogram='all', phase='all',
1657                       outputnames=None, makeBack=False):
1658        """Conducts one or a series of refinements according to the
1659           input provided in parameter refinements. This is a wrapper
1660           around :meth:`iter_refinements`
1661
1662        :param list refinements: A list of dictionaries specifiying changes to be made to
1663            parameters before refinements are conducted.
1664            See the :ref:`Refinement_recipe` section for how this is defined.
1665        :param str histogram: Name of histogram for refinements to be applied
1666            to, or 'all'; note that this can be overridden for each refinement
1667            step via a "histograms" entry in the dict.
1668        :param str phase: Name of phase for refinements to be applied to, or
1669            'all'; note that this can be overridden for each refinement
1670            step via a "phases" entry in the dict.
1671        :param list outputnames: Provides a list of project (.gpx) file names
1672            to use for each refinement step (specifying None skips the save step).
1673            See :meth:`save`.
1674            Note that this can be overridden using an "output" entry in the dict.
1675        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
1676            before a refinement is performed. The default is False.
1677           
1678        To perform a single refinement without changing any parameters, use this
1679        call:
1680
1681        .. code-block::  python
1682       
1683            my_project.do_refinements([])
1684        """
1685       
1686        for proj in self.iter_refinements(refinements, histogram, phase,
1687                                          outputnames, makeBack):
1688            pass
1689        return self
1690
1691    def iter_refinements(self, refinements, histogram='all', phase='all',
1692                         outputnames=None, makeBack=False):
1693        """Conducts a series of refinements, iteratively. Stops after every
1694        refinement and yields this project, to allow error checking or
1695        logging of intermediate results. Parameter use is the same as for
1696        :meth:`do_refinements` (which calls this method).
1697
1698        >>> def checked_refinements(proj):
1699        ...     for p in proj.iter_refinements(refs):
1700        ...         # Track intermediate results
1701        ...         log(p.histogram('0').residuals)
1702        ...         log(p.phase('0').get_cell())
1703        ...         # Check if parameter diverged, nonsense answer, or whatever
1704        ...         if is_something_wrong(p):
1705        ...             raise Exception("I need a human!")
1706
1707           
1708        """
1709        if outputnames:
1710            if len(refinements) != len(outputnames):
1711                raise ValueError("Should have same number of outputs to"
1712                                 "refinements")
1713        else:
1714            outputnames = [None for r in refinements]
1715
1716        for output, refinedict in zip(outputnames, refinements):
1717            if 'histograms' in refinedict:
1718                hist = refinedict['histograms']
1719            else:
1720                hist = histogram
1721            if 'phases' in refinedict:
1722                ph = refinedict['phases']
1723            else:
1724                ph = phase
1725            if 'output' in refinedict:
1726                output = refinedict['output']
1727            self.set_refinement(refinedict, hist, ph)
1728            # Handle 'once' args - refinements that are disabled after this
1729            # refinement
1730            if 'once' in refinedict:
1731                temp = {'set': refinedict['once']}
1732                self.set_refinement(temp, hist, ph)
1733
1734            if output:
1735                self.save(output)
1736
1737            if 'skip' not in refinedict:
1738                self.refine(makeBack=makeBack)
1739            yield self
1740
1741            # Handle 'once' args - refinements that are disabled after this
1742            # refinement
1743            if 'once' in refinedict:
1744                temp = {'clear': refinedict['once']}
1745                self.set_refinement(temp, hist, ph)
1746            if 'call' in refinedict:
1747                fxn = refinedict['call']
1748                if callable(fxn):
1749                    fxn(*refinedict.get('callargs',[self]))
1750                elif callable(eval(fxn)):
1751                    eval(fxn)(*refinedict.get('callargs',[self]))
1752                else:
1753                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
1754
1755    def set_refinement(self, refinement, histogram='all', phase='all'):
1756        """Apply specified refinements to a given histogram(s) or phase(s).
1757
1758        :param dict refinement: The refinements to be conducted
1759        :param histogram: Specifies either 'all' (default), a single histogram or
1760          a list of histograms. Histograms may be specified as histogram objects
1761          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
1762          of the histogram in the project, numbered starting from 0.
1763          Omitting the parameter or the string 'all' indicates that parameters in
1764          all histograms should be set.
1765        :param phase: Specifies either 'all' (default), a single phase or
1766          a list of phases. Phases may be specified as phase objects
1767          (see :class:`G2Phase`), the phase name (str) or the index number (int)
1768          of the phase in the project, numbered starting from 0.
1769          Omitting the parameter or the string 'all' indicates that parameters in
1770          all phases should be set.
1771
1772        Note that refinement parameters are categorized as one of three types:
1773
1774        1. Histogram parameters
1775        2. Phase parameters
1776        3. Histogram-and-Phase (HAP) parameters
1777       
1778        .. seealso::
1779            :meth:`G2PwdrData.set_refinements`
1780            :meth:`G2PwdrData.clear_refinements`
1781            :meth:`G2Phase.set_refinements`
1782            :meth:`G2Phase.clear_refinements`
1783            :meth:`G2Phase.set_HAP_refinements`
1784            :meth:`G2Phase.clear_HAP_refinements`"""
1785
1786        if histogram == 'all':
1787            hists = self.histograms()
1788        elif isinstance(histogram, list) or isinstance(histogram, tuple):
1789            hists = []
1790            for h in histogram:
1791                if isinstance(h, str) or isinstance(h, int):
1792                    hists.append(self.histogram(h))
1793                else:
1794                    hists.append(h)
1795        elif isinstance(histogram, str) or isinstance(histogram, int):
1796            hists = [self.histogram(histogram)]
1797        else:
1798            hists = [histogram]
1799
1800        if phase == 'all':
1801            phases = self.phases()
1802        elif isinstance(phase, list) or isinstance(phase, tuple):
1803            phases = []
1804            for ph in phase:
1805                if isinstance(ph, str) or isinstance(ph, int):
1806                    phases.append(self.phase(ph))
1807                else:
1808                    phases.append(ph)
1809        elif isinstance(phase, str) or isinstance(phase, int):
1810            phases = [self.phase(phase)]
1811        else:
1812            phases = [phase]
1813
1814        # TODO: HAP parameters:
1815        #   Babinet
1816        #   Extinction
1817        #   HStrain
1818        #   Mustrain
1819        #   Pref. Ori
1820        #   Size
1821
1822        pwdr_set = {}
1823        phase_set = {}
1824        hap_set = {}
1825        for key, val in refinement.get('set', {}).items():
1826            # Apply refinement options
1827            if G2PwdrData.is_valid_refinement_key(key):
1828                pwdr_set[key] = val
1829            elif G2Phase.is_valid_refinement_key(key):
1830                phase_set[key] = val
1831            elif G2Phase.is_valid_HAP_refinement_key(key):
1832                hap_set[key] = val
1833            else:
1834                raise ValueError("Unknown refinement key", key)
1835
1836        for hist in hists:
1837            hist.set_refinements(pwdr_set)
1838        for phase in phases:
1839            phase.set_refinements(phase_set)
1840        for phase in phases:
1841            phase.set_HAP_refinements(hap_set, hists)
1842
1843        pwdr_clear = {}
1844        phase_clear = {}
1845        hap_clear = {}
1846        for key, val in refinement.get('clear', {}).items():
1847            # Clear refinement options
1848            if G2PwdrData.is_valid_refinement_key(key):
1849                pwdr_clear[key] = val
1850            elif G2Phase.is_valid_refinement_key(key):
1851                phase_clear[key] = val
1852            elif G2Phase.is_valid_HAP_refinement_key(key):
1853                hap_set[key] = val
1854            else:
1855                raise ValueError("Unknown refinement key", key)
1856
1857        for hist in hists:
1858            hist.clear_refinements(pwdr_clear)
1859        for phase in phases:
1860            phase.clear_refinements(phase_clear)
1861        for phase in phases:
1862            phase.clear_HAP_refinements(hap_clear, hists)
1863
1864    def index_ids(self):
1865        import GSASIIstrIO as G2strIO
1866        self.save()
1867        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
1868
1869    def add_constraint_raw(self, cons_scope, constr):
1870        """Adds a constraint of type consType to the project.
1871        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
1872
1873        WARNING it does not check the constraint is well-constructed"""
1874        constrs = self.data['Constraints']['data']
1875        if 'Global' not in constrs:
1876            constrs['Global'] = []
1877        constrs[cons_scope].append(constr)
1878
1879    def hold_many(self, vars, type):
1880        """Apply holds for all the variables in vars, for constraint of a given type.
1881
1882        type is passed directly to add_constraint_raw as consType
1883
1884        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
1885            string variable specifiers, or arguments for :meth:`make_var_obj`
1886        :param str type: A string constraint type specifier. See
1887            :class:`G2Project.add_constraint_raw`
1888
1889        """
1890        for var in vars:
1891            if isinstance(var, str):
1892                var = self.make_var_obj(var)
1893            elif not isinstance(var, G2obj.G2VarObj):
1894                var = self.make_var_obj(*var)
1895            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
1896
1897    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
1898                     reloadIdx=True):
1899        """Wrapper to create a G2VarObj. Takes either a string representaiton ("p:h:name:a")
1900        or individual names of phase, histogram, varname, and atomId.
1901
1902        Automatically converts string phase, hist, or atom names into the ID required
1903        by G2VarObj."""
1904
1905        if reloadIdx:
1906            self.index_ids()
1907
1908        # If string representation, short circuit
1909        if hist is None and varname is None and atomId is None:
1910            if isinstance(phase, str) and ':' in phase:
1911                return G2obj.G2VarObj(phase)
1912
1913        # Get phase index
1914        phaseObj = None
1915        if isinstance(phase, G2Phase):
1916            phaseObj = phase
1917            phase = G2obj.PhaseRanIdLookup[phase.ranId]
1918        elif phase in self.data['Phases']:
1919            phaseObj = self.phase(phase)
1920            phaseRanId = phaseObj.ranId
1921            phase = G2obj.PhaseRanIdLookup[phaseRanId]
1922
1923        # Get histogram index
1924        if isinstance(hist, G2PwdrData):
1925            hist = G2obj.HistRanIdLookup[hist.ranId]
1926        elif hist in self.data:
1927            histRanId = self.histogram(hist).ranId
1928            hist = G2obj.HistRanIdLookup[histRanId]
1929
1930        # Get atom index (if any)
1931        if isinstance(atomId, G2AtomRecord):
1932            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
1933        elif phaseObj:
1934            atomObj = phaseObj.atom(atomId)
1935            if atomObj:
1936                atomRanId = atomObj.ranId
1937                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
1938
1939        return G2obj.G2VarObj(phase, hist, varname, atomId)
1940
1941
1942class G2AtomRecord(G2ObjectWrapper):
1943    """Wrapper for an atom record. Has convenient accessors via @property.
1944
1945
1946    Available accessors: label, type, refinement_flags, coordinates,
1947        occupancy, ranId, id, adp_flag, uiso
1948
1949    Example:
1950
1951    >>> atom = some_phase.atom("O3")
1952    >>> # We can access the underlying data format
1953    >>> atom.data
1954    ['O3', 'O-2', '', ... ]
1955    >>> # We can also use wrapper accessors
1956    >>> atom.coordinates
1957    (0.33, 0.15, 0.5)
1958    >>> atom.refinement_flags
1959    u'FX'
1960    >>> atom.ranId
1961    4615973324315876477
1962    >>> atom.occupancy
1963    1.0
1964    """
1965    def __init__(self, data, indices, proj):
1966        self.data = data
1967        self.cx, self.ct, self.cs, self.cia = indices
1968        self.proj = proj
1969
1970    @property
1971    def label(self):
1972        return self.data[self.ct-1]
1973
1974    @property
1975    def type(self):
1976        return self.data[self.ct]
1977
1978    @property
1979    def refinement_flags(self):
1980        return self.data[self.ct+1]
1981
1982    @refinement_flags.setter
1983    def refinement_flags(self, other):
1984        # Automatically check it is a valid refinement
1985        for c in other:
1986            if c not in ' FXU':
1987                raise ValueError("Invalid atom refinement: ", other)
1988        self.data[self.ct+1] = other
1989
1990    @property
1991    def coordinates(self):
1992        return tuple(self.data[self.cx:self.cx+3])
1993
1994    @property
1995    def occupancy(self):
1996        return self.data[self.cx+3]
1997
1998    @occupancy.setter
1999    def occupancy(self, val):
2000        self.data[self.cx+3] = float(val)
2001
2002    @property
2003    def ranId(self):
2004        return self.data[self.cia+8]
2005
2006    @property
2007    def adp_flag(self):
2008        # Either 'I' or 'A'
2009        return self.data[self.cia]
2010
2011    @property
2012    def uiso(self):
2013        if self.adp_flag == 'I':
2014            return self.data[self.cia+1]
2015        else:
2016            return self.data[self.cia+2:self.cia+8]
2017
2018    @uiso.setter
2019    def uiso(self, value):
2020        if self.adp_flag == 'I':
2021            self.data[self.cia+1] = float(value)
2022        else:
2023            assert len(value) == 6
2024            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2025
2026
2027class G2PwdrData(G2ObjectWrapper):
2028    """Wraps a Powder Data Histogram."""
2029    def __init__(self, data, proj):
2030        self.data = data
2031        self.proj = proj
2032
2033    @staticmethod
2034    def is_valid_refinement_key(key):
2035        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2036                      'Instrument Parameters']
2037        return key in valid_keys
2038
2039    @property
2040    def name(self):
2041        return self.data['data'][-1]
2042
2043    @property
2044    def ranId(self):
2045        return self.data['data'][0]['ranId']
2046
2047    @property
2048    def residuals(self):
2049        '''Provides a dictionary with with the R-factors for this histogram.
2050        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2051        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2052        '''
2053        data = self.data['data'][0]
2054        return {key: data[key] for key in data
2055                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2056                   or ':' in key}
2057
2058    @property
2059    def InstrumentParameters(self):
2060        '''Provides a dictionary with with the Instrument Parameters
2061        for this histogram.
2062        '''
2063        return self.data['Instrument Parameters'][0]
2064
2065    @property
2066    def SampleParameters(self):
2067        '''Provides a dictionary with with the Sample Parameters
2068        for this histogram.
2069        '''
2070        return self.data['Sample Parameters']
2071   
2072    @property
2073    def id(self):
2074        self.proj.update_ids()
2075        return self.data['data'][0]['hId']
2076
2077    @id.setter
2078    def id(self, val):
2079        self.data['data'][0]['hId'] = val
2080
2081    def fit_fixed_points(self):
2082        """Attempts to apply a background fit to the fixed points currently specified."""
2083        def SetInstParms(Inst):
2084            dataType = Inst['Type'][0]
2085            insVary = []
2086            insNames = []
2087            insVals = []
2088            for parm in Inst:
2089                insNames.append(parm)
2090                insVals.append(Inst[parm][1])
2091                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2092                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2093                        Inst[parm][2] = False
2094            instDict = dict(zip(insNames, insVals))
2095            instDict['X'] = max(instDict['X'], 0.01)
2096            instDict['Y'] = max(instDict['Y'], 0.01)
2097            if 'SH/L' in instDict:
2098                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2099            return dataType, instDict, insVary
2100
2101        bgrnd = self.data['Background']
2102
2103        # Need our fixed points in order
2104        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2105        X = [x for x, y in bgrnd[1]['FixedPoints']]
2106        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2107
2108        limits = self.data['Limits'][1]
2109        if X[0] > limits[0]:
2110            X = [limits[0]] + X
2111            Y = [Y[0]] + Y
2112        if X[-1] < limits[1]:
2113            X += [limits[1]]
2114            Y += [Y[-1]]
2115
2116        # Some simple lookups
2117        controls = self.proj['Controls']['data']
2118        inst, inst2 = self.data['Instrument Parameters']
2119        pwddata = self.data['data'][1]
2120
2121        # Construct the data for background fitting
2122        xBeg = np.searchsorted(pwddata[0], limits[0])
2123        xFin = np.searchsorted(pwddata[0], limits[1])
2124        xdata = pwddata[0][xBeg:xFin]
2125        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2126
2127        W = [1]*len(xdata)
2128        Z = [0]*len(xdata)
2129
2130        dataType, insDict, insVary = SetInstParms(inst)
2131        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2132
2133        # Do the fit
2134        data = np.array([xdata, ydata, W, Z, Z, Z])
2135        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2136                        prevVaryList=bakVary, controls=controls)
2137
2138        # Post-fit
2139        parmDict = {}
2140        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2141        parmDict.update(bakDict)
2142        parmDict.update(insDict)
2143        pwddata[3][xBeg:xFin] *= 0
2144        pwddata[5][xBeg:xFin] *= 0
2145        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2146
2147        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2148        # TODO update background
2149        self.proj.save()
2150
2151    def getdata(self,datatype):
2152        '''Provides access to the histogram data of the selected data type
2153
2154        :param str datatype: must be one of the following values (case is ignored)
2155       
2156           * 'X': the 2theta or TOF values for the pattern
2157           * 'Yobs': the observed intensity values
2158           * 'Yweight': the weights for each data point (1/sigma**2)
2159           * 'Ycalc': the computed intensity values
2160           * 'Background': the computed background values
2161           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2162
2163        :returns: an numpy MaskedArray with data values of the requested type
2164       
2165        '''
2166        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2167        if datatype.lower() not in enums:
2168            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2169        return self.data['data'][1][enums.index(datatype.lower())]
2170       
2171    def y_calc(self):
2172        return self.data['data'][1][3]
2173
2174    def Export(self,fileroot,extension):
2175        '''Write the histogram into a file. The path is specified by fileroot and
2176        extension.
2177       
2178        :param str fileroot: name of the file, optionally with a path (extension is
2179           ignored)
2180        :param str extension: includes '.', must match an extension in global
2181           exportersByExtension['powder'] or a Exception is raised.
2182        :returns: name of file that was written
2183        '''
2184        if extension not in exportersByExtension.get('powder',[]):
2185            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2186        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2187        obj = exportersByExtension['powder'][extension]
2188        obj.SetFromArray(hist=self.data,histname=self.name)
2189        obj.Writer(self.name,fil)
2190           
2191    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2192        try:
2193            import matplotlib.pyplot as plt
2194            data = self.data['data'][1]
2195            if Yobs:
2196                plt.plot(data[0], data[1], label='Yobs')
2197            if Ycalc:
2198                plt.plot(data[0], data[3], label='Ycalc')
2199            if Background:
2200                plt.plot(data[0], data[4], label='Background')
2201            if Residual:
2202                plt.plot(data[0], data[5], label="Residual")
2203        except ImportError:
2204            pass
2205
2206    def get_wR(self):
2207        """returns the overall weighted profile R factor for a histogram
2208       
2209        :returns: a wR value as a percentage or None if not defined
2210        """
2211        return self['data'][0].get('wR')
2212
2213    def set_refinements(self, refs):
2214        """Sets the refinement parameter 'key' to the specification 'value'
2215
2216        :param dict refs: A dictionary of the parameters to be set. See
2217                          :ref:`Histogram_parameters_table` for a description of
2218                          what these dictionaries should be.
2219
2220        :returns: None
2221
2222        """
2223        do_fit_fixed_points = False
2224        for key, value in refs.items():
2225            if key == 'Limits':
2226                old_limits = self.data['Limits'][1]
2227                new_limits = value
2228                if isinstance(new_limits, dict):
2229                    if 'low' in new_limits:
2230                        old_limits[0] = new_limits['low']
2231                    if 'high' in new_limits:
2232                        old_limits[1] = new_limits['high']
2233                else:
2234                    old_limits[0], old_limits[1] = new_limits
2235            elif key == 'Sample Parameters':
2236                sample = self.data['Sample Parameters']
2237                for sparam in value:
2238                    if sparam not in sample:
2239                        raise ValueError("Unknown refinement parameter, "
2240                                         + str(sparam))
2241                    sample[sparam][1] = True
2242            elif key == 'Background':
2243                bkg, peaks = self.data['Background']
2244
2245                # If True or False, just set the refine parameter
2246                if value in (True, False):
2247                    bkg[1] = value
2248                    return
2249
2250                if 'type' in value:
2251                    bkg[0] = value['type']
2252                if 'refine' in value:
2253                    bkg[1] = value['refine']
2254                if 'no. coeffs' in value:
2255                    cur_coeffs = bkg[2]
2256                    n_coeffs = value['no. coeffs']
2257                    if n_coeffs > cur_coeffs:
2258                        for x in range(n_coeffs - cur_coeffs):
2259                            bkg.append(0.0)
2260                    else:
2261                        for _ in range(cur_coeffs - n_coeffs):
2262                            bkg.pop()
2263                    bkg[2] = n_coeffs
2264                if 'coeffs' in value:
2265                    bkg[3:] = value['coeffs']
2266                if 'FixedPoints' in value:
2267                    peaks['FixedPoints'] = [(float(a), float(b))
2268                                            for a, b in value['FixedPoints']]
2269                if value.get('fit fixed points', False):
2270                    do_fit_fixed_points = True
2271
2272            elif key == 'Instrument Parameters':
2273                instrument, secondary = self.data['Instrument Parameters']
2274                for iparam in value:
2275                    try:
2276                        instrument[iparam][2] = True
2277                    except IndexError:
2278                        raise ValueError("Invalid key:", iparam)
2279            else:
2280                raise ValueError("Unknown key:", key)
2281        # Fit fixed points after the fact - ensure they are after fixed points
2282        # are added
2283        if do_fit_fixed_points:
2284            # Background won't be fit if refinement flag not set
2285            orig = self.data['Background'][0][1]
2286            self.data['Background'][0][1] = True
2287            self.fit_fixed_points()
2288            # Restore the previous value
2289            self.data['Background'][0][1] = orig
2290
2291    def clear_refinements(self, refs):
2292        """Clears the refinement parameter 'key' and its associated value.
2293
2294        :param dict refs: A dictionary of parameters to clear."""
2295        for key, value in refs.items():
2296            if key == 'Limits':
2297                old_limits, cur_limits = self.data['Limits']
2298                cur_limits[0], cur_limits[1] = old_limits
2299            elif key == 'Sample Parameters':
2300                sample = self.data['Sample Parameters']
2301                for sparam in value:
2302                    sample[sparam][1] = False
2303            elif key == 'Background':
2304                bkg, peaks = self.data['Background']
2305
2306                # If True or False, just set the refine parameter
2307                if value in (True, False):
2308                    bkg[1] = False
2309                    return
2310
2311                bkg[1] = False
2312                if 'FixedPoints' in value:
2313                    if 'FixedPoints' in peaks:
2314                        del peaks['FixedPoints']
2315            elif key == 'Instrument Parameters':
2316                instrument, secondary = self.data['Instrument Parameters']
2317                for iparam in value:
2318                    instrument[iparam][2] = False
2319            else:
2320                raise ValueError("Unknown key:", key)
2321
2322    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2323        '''Adds a single peak to the peak list
2324        :param float area: peak area
2325        :param float dspace: peak position as d-space (A)
2326        :param float Q: peak position as Q (A-1)
2327        :param float ttheta: peak position as 2Theta (deg)
2328
2329        Note: only one of the parameters dspace, Q or ttheta may be specified
2330        '''
2331        import GSASIIlattice as G2lat
2332        import GSASIImath as G2mth
2333        if (not dspace) + (not Q) + (not ttheta) != 2:
2334            print('add_peak error: too many or no peak position(s) specified')
2335            return
2336        pos = ttheta
2337        Parms,Parms2 = self.data['Instrument Parameters']
2338        if Q:
2339            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2340        elif dspace:
2341            pos = G2lat.Dsp2pos(Parms,dspace)
2342        peaks = self.data['Peak List']
2343        peaks['sigDict'] = {}        #no longer valid
2344        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2345
2346    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2347        '''Set refinement flags for peaks
2348        :param list peaklist: a list of peaks to change flags. If None (default), changes
2349          are made to all peaks.
2350        :param bool area: Sets or clears the refinement flag for the peak area value.
2351          If None (the default), no change is made.
2352        :param bool pos: Sets or clears the refinement flag for the peak position value.
2353          If None (the default), no change is made.
2354        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2355          If None (the default), no change is made.
2356        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2357          If None (the default), no change is made.
2358         
2359        Note that when peaks are first created the area flag is on and the other flags are
2360        initially off.
2361
2362        Example::
2363       
2364           set_peakFlags(sig=False,gam=True)
2365
2366        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
2367        all peaks. The position and area flags are not changed from their previous values.
2368        '''
2369        peaks = self.data['Peak List']
2370        if peaklist is None:
2371            peaklist = range(len(peaks['peaks']))
2372        for i in peaklist:
2373            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
2374                if var is not None:
2375                    peaks['peaks'][i][j] = var
2376           
2377    def refine_peaks(self):
2378        '''Causes a refinement of peak position, background and instrument parameters
2379        '''
2380        import GSASIIpwd as G2pwd
2381        controls = self.proj.data.get('Controls',{})
2382        controls = controls.get('data',
2383                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
2384                            )
2385        peaks = self.data['Peak List']
2386        Parms,Parms2 = self.data['Instrument Parameters']
2387        background = self.data['Background']
2388        limits = self.data['Limits'][1]
2389        bxye = np.zeros(len(self.data['data'][1][1]))
2390        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
2391                                           Parms,Parms2,self.data['data'][1],bxye,[],
2392                                           False,controls,None)[0]
2393
2394    def SaveProfile(self,filename):
2395        '''Writes a GSAS-II (new style) .instprm file
2396        '''
2397        data,Parms2 = self.data['Instrument Parameters']
2398        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2399        File = open(filename,'w')
2400        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
2401        for item in data:
2402            File.write(item+':'+str(data[item][1])+'\n')
2403        File.close()
2404        print ('Instrument parameters saved to: '+filename)
2405
2406    def LoadProfile(self,filename):
2407        '''Reads a GSAS-II (new style) .instprm file and overwrites the current parameters
2408        '''
2409        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2410        File = open(filename,'r')
2411        S = File.readline()
2412        newItems = []
2413        newVals = []
2414        Found = False
2415        while S:
2416            if S[0] == '#':
2417                if Found:
2418                    break
2419                if 'Bank' in S:
2420                    if bank == int(S.split(':')[0].split()[1]):
2421                        S = File.readline()
2422                        continue
2423                    else:
2424                        S = File.readline()
2425                        while S and '#Bank' not in S:
2426                            S = File.readline()
2427                        continue
2428                else:   #a non #Bank file
2429                    S = File.readline()
2430                    continue
2431            Found = True
2432            [item,val] = S[:-1].split(':')
2433            newItems.append(item)
2434            try:
2435                newVals.append(float(val))
2436            except ValueError:
2437                newVals.append(val)                       
2438            S = File.readline()               
2439        File.close()
2440        LoadG2fil()
2441        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
2442
2443       
2444class G2Phase(G2ObjectWrapper):
2445    """A wrapper object around a given phase.
2446
2447    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
2448    """
2449    def __init__(self, data, name, proj):
2450        self.data = data
2451        self.name = name
2452        self.proj = proj
2453
2454    @staticmethod
2455    def is_valid_refinement_key(key):
2456        valid_keys = ["Cell", "Atoms", "LeBail"]
2457        return key in valid_keys
2458
2459    @staticmethod
2460    def is_valid_HAP_refinement_key(key):
2461        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
2462                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
2463        return key in valid_keys
2464
2465    def atom(self, atomlabel):
2466        """Returns the atom specified by atomlabel, or None if it does not
2467        exist.
2468
2469        :param str atomlabel: The name of the atom (e.g. "O2")
2470        :returns: A :class:`G2AtomRecord` object
2471            representing the atom.
2472        """
2473        # Consult GSASIIobj.py for the meaning of this
2474        cx, ct, cs, cia = self.data['General']['AtomPtrs']
2475        ptrs = [cx, ct, cs, cia]
2476        atoms = self.data['Atoms']
2477        for atom in atoms:
2478            if atom[ct-1] == atomlabel:
2479                return G2AtomRecord(atom, ptrs, self.proj)
2480
2481    def atoms(self):
2482        """Returns a list of atoms present in the phase.
2483
2484        :returns: A list of :class:`G2AtomRecord` objects.
2485
2486        .. seealso::
2487            :meth:`G2Phase.atom`
2488            :class:`G2AtomRecord`
2489        """
2490        ptrs = self.data['General']['AtomPtrs']
2491        output = []
2492        atoms = self.data['Atoms']
2493        for atom in atoms:
2494            output.append(G2AtomRecord(atom, ptrs, self.proj))
2495        return output
2496
2497    def histograms(self):
2498        output = []
2499        for hname in self.data.get('Histograms', {}).keys():
2500            output.append(self.proj.histogram(hname))
2501        return output
2502
2503    @property
2504    def ranId(self):
2505        return self.data['ranId']
2506
2507    @property
2508    def id(self):
2509        return self.data['pId']
2510
2511    @id.setter
2512    def id(self, val):
2513        self.data['pId'] = val
2514
2515    def get_cell(self):
2516        """Returns a dictionary of the cell parameters, with keys:
2517            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
2518
2519        :returns: a dict
2520
2521        .. seealso::
2522           :meth:`G2Phase.get_cell_and_esd`
2523
2524        """
2525        cell = self.data['General']['Cell']
2526        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
2527                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
2528                'volume': cell[7]}
2529
2530    def get_cell_and_esd(self):
2531        """
2532        Returns a pair of dictionaries, the first representing the unit cell, the second
2533        representing the estimated standard deviations of the unit cell.
2534
2535        :returns: a tuple of two dictionaries
2536
2537        .. seealso::
2538           :meth:`G2Phase.get_cell`
2539
2540        """
2541        # translated from GSASIIstrIO.ExportBaseclass.GetCell
2542        import GSASIIstrIO as G2stIO
2543        import GSASIIlattice as G2lat
2544        import GSASIImapvars as G2mv
2545        try:
2546            pfx = str(self.id) + '::'
2547            sgdata = self['General']['SGData']
2548            covDict = self.proj['Covariance']['data']
2549
2550            parmDict = dict(zip(covDict.get('varyList',[]),
2551                                covDict.get('variables',[])))
2552            sigDict = dict(zip(covDict.get('varyList',[]),
2553                               covDict.get('sig',[])))
2554
2555            if covDict.get('covMatrix') is not None:
2556                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2557                                                  covDict['varyList'],
2558                                                  parmDict))
2559
2560            A, sigA = G2stIO.cellFill(pfx, sgdata, parmDict, sigDict)
2561            cellSig = G2stIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
2562            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
2563            cellDict, cellSigDict = {}, {}
2564            for i, key in enumerate(['length_a', 'length_b', 'length_c',
2565                                     'angle_alpha', 'angle_beta', 'angle_gamma',
2566                                     'volume']):
2567                cellDict[key] = cellList[i]
2568                cellSigDict[key] = cellSig[i]
2569            return cellDict, cellSigDict
2570        except KeyError:
2571            cell = self.get_cell()
2572            return cell, {key: 0.0 for key in cell}
2573
2574    def export_CIF(self, outputname, quickmode=True):
2575        """Write this phase to a .cif file named outputname
2576
2577        :param str outputname: The name of the .cif file to write to
2578        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
2579        # This code is all taken from exports/G2export_CIF.py
2580        # Functions copied have the same names
2581        import GSASIImath as G2mth
2582        import GSASIImapvars as G2mv
2583        from exports import G2export_CIF as cif
2584
2585        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
2586        CIFname = os.path.splitext(self.proj.filename)[0]
2587        CIFname = os.path.split(CIFname)[1]
2588        CIFname = ''.join([c if ord(c) < 128 else ''
2589                           for c in CIFname.replace(' ', '_')])
2590        try:
2591            author = self.proj['Controls']['data'].get('Author','').strip()
2592        except KeyError:
2593            pass
2594        oneblock = True
2595
2596        covDict = self.proj['Covariance']['data']
2597        parmDict = dict(zip(covDict.get('varyList',[]),
2598                            covDict.get('variables',[])))
2599        sigDict = dict(zip(covDict.get('varyList',[]),
2600                           covDict.get('sig',[])))
2601
2602        if covDict.get('covMatrix') is not None:
2603            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2604                                              covDict['varyList'],
2605                                              parmDict))
2606
2607        with open(outputname, 'w') as fp:
2608            fp.write(' \n' + 70*'#' + '\n')
2609            cif.WriteCIFitem(fp, 'data_' + CIFname)
2610            # from exports.G2export_CIF.WritePhaseInfo
2611            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
2612            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
2613            # TODO get esds
2614            cellDict = self.get_cell()
2615            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
2616            names = ['length_a','length_b','length_c',
2617                     'angle_alpha','angle_beta ','angle_gamma',
2618                     'volume']
2619            for key, val in cellDict.items():
2620                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
2621
2622            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
2623                         self.data['General']['SGData']['SGSys'])
2624
2625            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
2626            # regularize capitalization and remove trailing H/R
2627            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
2628            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
2629
2630            # generate symmetry operations including centering and center of symmetry
2631            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
2632                self.data['General']['SGData'])
2633            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
2634            for i, op in enumerate(SymOpList,start=1):
2635                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
2636
2637            # TODO skipped histograms, exports/G2export_CIF.py:880
2638
2639            # report atom params
2640            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
2641                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
2642                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
2643            else:
2644                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
2645            # report cell contents
2646            cif.WriteComposition(fp, self.data, self.name, parmDict)
2647            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
2648                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
2649                raise NotImplementedError("only quickmode currently supported")
2650            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
2651                cif.WriteCIFitem(fp,'\n# Difference density results')
2652                MinMax = self.data['General']['Map']['minmax']
2653                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
2654                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
2655
2656
2657    def set_refinements(self, refs):
2658        """Sets the refinement parameter 'key' to the specification 'value'
2659
2660        :param dict refs: A dictionary of the parameters to be set. See
2661                          :ref:`Phase_parameters_table` for a description of
2662                          this dictionary.
2663
2664        :returns: None"""
2665        for key, value in refs.items():
2666            if key == "Cell":
2667                self.data['General']['Cell'][0] = value
2668
2669            elif key == "Atoms":
2670                for atomlabel, atomrefinement in value.items():
2671                    if atomlabel == 'all':
2672                        for atom in self.atoms():
2673                            atom.refinement_flags = atomrefinement
2674                    else:
2675                        atom = self.atom(atomlabel)
2676                        if atom is None:
2677                            raise ValueError("No such atom: " + atomlabel)
2678                        atom.refinement_flags = atomrefinement
2679
2680            elif key == "LeBail":
2681                hists = self.data['Histograms']
2682                for hname, hoptions in hists.items():
2683                    if 'LeBail' not in hoptions:
2684                        hoptions['newLeBail'] = bool(True)
2685                    hoptions['LeBail'] = bool(value)
2686            else:
2687                raise ValueError("Unknown key:", key)
2688
2689    def clear_refinements(self, refs):
2690        """Clears a given set of parameters.
2691
2692        :param dict refs: The parameters to clear"""
2693        for key, value in refs.items():
2694            if key == "Cell":
2695                self.data['General']['Cell'][0] = False
2696            elif key == "Atoms":
2697                cx, ct, cs, cia = self.data['General']['AtomPtrs']
2698
2699                for atomlabel in value:
2700                    atom = self.atom(atomlabel)
2701                    # Set refinement to none
2702                    atom.refinement_flags = ' '
2703            elif key == "LeBail":
2704                hists = self.data['Histograms']
2705                for hname, hoptions in hists.items():
2706                    if 'LeBail' not in hoptions:
2707                        hoptions['newLeBail'] = True
2708                    hoptions['LeBail'] = False
2709            else:
2710                raise ValueError("Unknown key:", key)
2711
2712    def set_HAP_refinements(self, refs, histograms='all'):
2713        """Sets the given HAP refinement parameters between this phase and
2714        the given histograms
2715
2716        :param dict refs: A dictionary of the parameters to be set. See
2717                          :ref:`HAP_parameters_table` for a description of this
2718                          dictionary.
2719        :param histograms: Either 'all' (default) or a list of the histograms
2720            whose HAP parameters will be set with this phase. Histogram and phase
2721            must already be associated
2722
2723        :returns: None
2724        """
2725        if histograms == 'all':
2726            histograms = self.data['Histograms'].values()
2727        else:
2728            histograms = [self.data['Histograms'][h.name] for h in histograms]
2729
2730        for key, val in refs.items():
2731            for h in histograms:
2732                if key == 'Babinet':
2733                    try:
2734                        sets = list(val)
2735                    except ValueError:
2736                        sets = ['BabA', 'BabU']
2737                    for param in sets:
2738                        if param not in ['BabA', 'BabU']:
2739                            raise ValueError("Not sure what to do with" + param)
2740                        for hist in histograms:
2741                            hist['Babinet'][param][1] = True
2742                elif key == 'Extinction':
2743                    for h in histograms:
2744                        h['Extinction'][1] = bool(val)
2745                elif key == 'HStrain':
2746                    for h in histograms:
2747                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
2748                elif key == 'Mustrain':
2749                    for h in histograms:
2750                        mustrain = h['Mustrain']
2751                        newType = None
2752                        direction = None
2753                        if isinstance(val, strtypes):
2754                            if val in ['isotropic', 'uniaxial', 'generalized']:
2755                                newType = val
2756                            else:
2757                                raise ValueError("Not a Mustrain type: " + val)
2758                        elif isinstance(val, dict):
2759                            newType = val.get('type', None)
2760                            direction = val.get('direction', None)
2761
2762                        if newType:
2763                            mustrain[0] = newType
2764                            if newType == 'isotropic':
2765                                mustrain[2][0] = True == val.get('refine',False)
2766                                mustrain[5] = [False for p in mustrain[4]]
2767                            elif newType == 'uniaxial':
2768                                if 'refine' in val:
2769                                    mustrain[2][0] = False
2770                                    types = val['refine']
2771                                    if isinstance(types, strtypes):
2772                                        types = [types]
2773                                    elif isinstance(types, bool):
2774                                        mustrain[2][1] = types
2775                                        mustrain[2][2] = types
2776                                        types = []
2777                                    else:
2778                                        raise ValueError("Not sure what to do with: "
2779                                                         + str(types))
2780                                else:
2781                                    types = []
2782
2783                                for unitype in types:
2784                                    if unitype == 'equatorial':
2785                                        mustrain[2][0] = True
2786                                    elif unitype == 'axial':
2787                                        mustrain[2][1] = True
2788                                    else:
2789                                        msg = 'Invalid uniaxial mustrain type'
2790                                        raise ValueError(msg + ': ' + unitype)
2791                            else:  # newtype == 'generalized'
2792                                mustrain[2] = [False for p in mustrain[1]]
2793                                if 'refine' in val:
2794                                    mustrain[5] = [True == val['refine']]*len(mustrain[5])
2795
2796                        if direction:
2797                            if len(direction) != 3:
2798                                raise ValueError("Expected hkl, found", direction)
2799                            direction = [int(n) for n in direction]
2800                            mustrain[3] = direction
2801                elif key == 'Size':
2802                    newSize = None
2803                    if 'value' in val:
2804                        newSize = float(val['value'])
2805                    for h in histograms:
2806                        size = h['Size']
2807                        newType = None
2808                        direction = None
2809                        if isinstance(val, strtypes):
2810                            if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
2811                                newType = val
2812                            else:
2813                                raise ValueError("Not a valid Size type: " + val)
2814                        elif isinstance(val, dict):
2815                            newType = val.get('type', None)
2816                            direction = val.get('direction', None)
2817
2818                        if newType:
2819                            size[0] = newType
2820                            refine = True == val.get('refine')
2821                            if newType == 'isotropic' and refine is not None:
2822                                size[2][0] = bool(refine)
2823                                if newSize: size[1][0] = newSize
2824                            elif newType == 'uniaxial' and refine is not None:
2825                                size[2][1] = bool(refine)
2826                                size[2][2] = bool(refine)
2827                                if newSize: size[1][1] = size[1][2] =newSize
2828                            elif newType == 'ellipsoidal' and refine is not None:
2829                                size[5] = [bool(refine) for p in size[5]]
2830                                if newSize: size[4] = [newSize for p in size[4]]
2831
2832                        if direction:
2833                            if len(direction) != 3:
2834                                raise ValueError("Expected hkl, found", direction)
2835                            direction = [int(n) for n in direction]
2836                            size[3] = direction
2837                elif key == 'Pref.Ori.':
2838                    for h in histograms:
2839                        h['Pref.Ori.'][2] = bool(val)
2840                elif key == 'Show':
2841                    for h in histograms:
2842                        h['Show'] = bool(val)
2843                elif key == 'Use':
2844                    for h in histograms:
2845                        h['Use'] = bool(val)
2846                elif key == 'Scale':
2847                    for h in histograms:
2848                        h['Scale'][1] = bool(val)
2849                else:
2850                    print(u'Unknown HAP key: '+key)
2851
2852    def getHAPvalues(self, histname):
2853        """Returns a dict with HAP values for the selected histogram
2854
2855        :param histogram: is a histogram object (:class:`G2PwdrData`) or
2856            a histogram name or the index number of the histogram
2857
2858        :returns: HAP value dict
2859        """
2860        if isinstance(histname, G2PwdrData):
2861            histname = histname.name
2862        elif histname in self.data['Histograms']:
2863            pass
2864        elif type(histname) is int:
2865            histname = self.proj.histograms()[histname].name
2866        else:
2867            raise G2ScriptException("Invalid histogram reference: "+str(histname))
2868        return self.data['Histograms'][histname]
2869                   
2870    def clear_HAP_refinements(self, refs, histograms='all'):
2871        """Clears the given HAP refinement parameters between this phase and
2872        the given histograms
2873
2874        :param dict refs: A dictionary of the parameters to be cleared.
2875        :param histograms: Either 'all' (default) or a list of the histograms
2876            whose HAP parameters will be cleared with this phase. Histogram and
2877            phase must already be associated
2878
2879        :returns: None
2880        """
2881        if histograms == 'all':
2882            histograms = self.data['Histograms'].values()
2883        else:
2884            histograms = [self.data['Histograms'][h.name] for h in histograms]
2885
2886        for key, val in refs.items():
2887            for h in histograms:
2888                if key == 'Babinet':
2889                    try:
2890                        sets = list(val)
2891                    except ValueError:
2892                        sets = ['BabA', 'BabU']
2893                    for param in sets:
2894                        if param not in ['BabA', 'BabU']:
2895                            raise ValueError("Not sure what to do with" + param)
2896                        for hist in histograms:
2897                            hist['Babinet'][param][1] = False
2898                elif key == 'Extinction':
2899                    for h in histograms:
2900                        h['Extinction'][1] = False
2901                elif key == 'HStrain':
2902                    for h in histograms:
2903                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
2904                elif key == 'Mustrain':
2905                    for h in histograms:
2906                        mustrain = h['Mustrain']
2907                        mustrain[2] = [False for p in mustrain[2]]
2908                        mustrain[5] = [False for p in mustrain[4]]
2909                elif key == 'Pref.Ori.':
2910                    for h in histograms:
2911                        h['Pref.Ori.'][2] = False
2912                elif key == 'Show':
2913                    for h in histograms:
2914                        h['Show'] = False
2915                elif key == 'Size':
2916                    for h in histograms:
2917                        size = h['Size']
2918                        size[2] = [False for p in size[2]]
2919                        size[5] = [False for p in size[5]]
2920                elif key == 'Use':
2921                    for h in histograms:
2922                        h['Use'] = False
2923                elif key == 'Scale':
2924                    for h in histograms:
2925                        h['Scale'][1] = False
2926                else:
2927                    print(u'Unknown HAP key: '+key)
2928
2929
2930##########################
2931# Command Line Interface #
2932##########################
2933# Each of these takes an argparse.Namespace object as their argument,
2934# representing the parsed command-line arguments for the relevant subcommand.
2935# The argument specification for each is in the subcommands dictionary (see
2936# below)
2937
2938commandhelp={}
2939commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
2940def create(args):
2941    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
2942
2943  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2944                                  [-i IPARAMS [IPARAMS ...]]
2945                                  [-p PHASES [PHASES ...]]
2946                                  filename
2947                                 
2948positional arguments::
2949
2950  filename              the project file to create. should end in .gpx
2951
2952optional arguments::
2953
2954  -h, --help            show this help message and exit
2955  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2956                        list of datafiles to add as histograms
2957  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2958                        instrument parameter file, must be one for every
2959                        histogram
2960  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2961                        list of phases to add. phases are automatically
2962                        associated with all histograms given.
2963
2964    """
2965    proj = G2Project(gpxname=args.filename)
2966
2967    hist_objs = []
2968    if args.histograms:
2969        for h,i in zip(args.histograms,args.iparams):
2970            print("Adding histogram from",h,"with instparm ",i)
2971            hist_objs.append(proj.add_powder_histogram(h, i))
2972
2973    if args.phases: 
2974        for p in args.phases:
2975            print("Adding phase from",p)
2976            proj.add_phase(p, histograms=hist_objs)
2977        print('Linking phase(s) to histogram(s):')
2978        for h in hist_objs:
2979            print ('   '+h.name)
2980
2981    proj.save()
2982
2983commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
2984def add(args):
2985    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
2986
2987  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2988                               [-i IPARAMS [IPARAMS ...]]
2989                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
2990                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
2991                               filename
2992
2993
2994positional arguments::
2995
2996  filename              the project file to open. Should end in .gpx
2997
2998optional arguments::
2999
3000  -h, --help            show this help message and exit
3001  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
3002                        list of datafiles to add as histograms
3003  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
3004                        instrument parameter file, must be one for every
3005                        histogram
3006  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
3007                        format hint for histogram import. Applies to all
3008                        histograms
3009  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
3010                        list of phases to add. phases are automatically
3011                        associated with all histograms given.
3012  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
3013                        format hint for phase import. Applies to all phases.
3014                        Example: -pf CIF
3015  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
3016                        list of histgram indices to associate with added
3017                        phases. If not specified, phases are associated with
3018                        all previously loaded histograms. Example: -l 2 3 4
3019   
3020    """
3021    proj = G2Project(args.filename)
3022
3023    if args.histograms:
3024        for h,i in zip(args.histograms,args.iparams):
3025            print("Adding histogram from",h,"with instparm ",i)
3026            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
3027
3028    if args.phases: 
3029        if not args.histlist:
3030            histlist = proj.histograms()
3031        else:
3032            histlist = [proj.histogram(i) for i in args.histlist]
3033
3034        for p in args.phases:
3035            print("Adding phase from",p)
3036            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
3037           
3038        if not args.histlist:
3039            print('Linking phase(s) to all histogram(s)')
3040        else:
3041            print('Linking phase(s) to histogram(s):')
3042            for h in histlist:
3043                print ('   '+h.name)
3044
3045    proj.save()
3046
3047
3048commandhelp["dump"] = "Shows the contents of a GSAS-II project"
3049def dump(args):
3050    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
3051
3052       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
3053
3054positional arguments::
3055
3056  files
3057
3058optional arguments::
3059
3060  -h, --help        show this help message and exit
3061  -d, --histograms  list histograms in files, overrides --raw
3062  -p, --phases      list phases in files, overrides --raw
3063  -r, --raw         dump raw file contents, default
3064 
3065    """
3066    if not args.histograms and not args.phases:
3067        args.raw = True
3068    if args.raw:
3069        import IPython.lib.pretty as pretty
3070
3071    for fname in args.files:
3072        if args.raw:
3073            proj, nameList = LoadDictFromProjFile(fname)
3074            print("file:", fname)
3075            print("names:", nameList)
3076            for key, val in proj.items():
3077                print(key, ":")
3078                pretty.pprint(val)
3079        else:
3080            proj = G2Project(fname)
3081            if args.histograms:
3082                hists = proj.histograms()
3083                for h in hists:
3084                    print(fname, "hist", h.id, h.name)
3085            if args.phases:
3086                phase_list = proj.phases()
3087                for p in phase_list:
3088                    print(fname, "phase", p.id, p.name)
3089
3090
3091commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
3092def IPyBrowse(args):
3093    """Load a .gpx file and then open a IPython shell to browse it::
3094
3095  usage: GSASIIscriptable.py browse [-h] files [files ...]
3096
3097positional arguments::
3098
3099  files       list of files to browse
3100
3101optional arguments::
3102
3103  -h, --help  show this help message and exit
3104
3105    """
3106    for fname in args.files:
3107        proj, nameList = LoadDictFromProjFile(fname)
3108        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
3109        GSASIIpath.IPyBreak_base(msg)
3110        break
3111
3112
3113commandhelp["refine"] = '''
3114Conducts refinements on GSAS-II projects according to a list of refinement
3115steps in a JSON dict
3116'''
3117def refine(args):
3118    """Implements the refine command-line subcommand:
3119    conducts refinements on GSAS-II projects according to a JSON refinement dict::
3120
3121        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
3122
3123positional arguments::
3124
3125  gpxfile      the project file to refine
3126  refinements  json file of refinements to apply. if not present refines file
3127               as-is
3128
3129optional arguments::
3130
3131  -h, --help   show this help message and exit
3132 
3133    """
3134    proj = G2Project(args.gpxfile)
3135    if args.refinements is None:
3136        proj.refine()
3137    else:
3138        import json
3139        with open(args.refinements) as refs:
3140            refs = json.load(refs)
3141        if type(refs) is not dict:
3142            raise G2ScriptException("Error: JSON object must be a dict.")
3143        if "code" in refs:
3144            print("executing code:\n|  ",'\n|  '.join(refs['code']))
3145            exec('\n'.join(refs['code']))
3146        proj.do_refinements(refs['refinements'])
3147
3148
3149commandhelp["seqrefine"] = "Not implemented. Placeholder for eventual sequential refinement implementation"
3150def seqrefine(args):
3151    """Future implementation for the seqrefine command-line subcommand """
3152    raise NotImplementedError("seqrefine is not yet implemented")
3153
3154
3155commandhelp["export"] = "Export phase as CIF"
3156def export(args):
3157    """Implements the export command-line subcommand: Exports phase as CIF::
3158
3159      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
3160
3161positional arguments::
3162
3163  gpxfile     the project file from which to export
3164  phase       identifier of phase to export
3165  exportfile  the .cif file to export to
3166
3167optional arguments::
3168
3169  -h, --help  show this help message and exit
3170
3171    """
3172    proj = G2Project(args.gpxfile)
3173    phase = proj.phase(args.phase)
3174    phase.export_CIF(args.exportfile)
3175
3176
3177def _args_kwargs(*args, **kwargs):
3178    return args, kwargs
3179
3180# A dictionary of the name of each subcommand, and a tuple
3181# of its associated function and a list of its arguments
3182# The arguments are passed directly to the add_argument() method
3183# of an argparse.ArgumentParser
3184
3185subcommands = {"create":
3186               (create, [_args_kwargs('filename',
3187                                      help='the project file to create. should end in .gpx'),
3188
3189                         _args_kwargs('-d', '--histograms',
3190                                      nargs='+',
3191                                      help='list of datafiles to add as histograms'),
3192                                     
3193                         _args_kwargs('-i', '--iparams',
3194                                      nargs='+',
3195                                      help='instrument parameter file, must be one'
3196                                           ' for every histogram'
3197                                      ),
3198
3199                         _args_kwargs('-p', '--phases',
3200                                      nargs='+',
3201                                      help='list of phases to add. phases are '
3202                                           'automatically associated with all '
3203                                           'histograms given.')]),
3204               "add": (add, [_args_kwargs('filename',
3205                                      help='the project file to open. Should end in .gpx'),
3206
3207                         _args_kwargs('-d', '--histograms',
3208                                      nargs='+',
3209                                      help='list of datafiles to add as histograms'),
3210                                     
3211                         _args_kwargs('-i', '--iparams',
3212                                      nargs='+',
3213                                      help='instrument parameter file, must be one'
3214                                           ' for every histogram'
3215                                      ),
3216                                     
3217                         _args_kwargs('-hf', '--histogramformat',
3218                                      help='format hint for histogram import. Applies to all'
3219                                           ' histograms'
3220                                      ),
3221
3222                         _args_kwargs('-p', '--phases',
3223                                      nargs='+',
3224                                      help='list of phases to add. phases are '
3225                                           'automatically associated with all '
3226                                           'histograms given.'),
3227
3228                         _args_kwargs('-pf', '--phaseformat',
3229                                      help='format hint for phase import. Applies to all'
3230                                           ' phases. Example: -pf CIF'
3231                                      ),
3232                                     
3233                         _args_kwargs('-l', '--histlist',
3234                                      nargs='+',
3235                                      help='list of histgram indices to associate with added'
3236                                           ' phases. If not specified, phases are'
3237                                           ' associated with all previously loaded'
3238                                           ' histograms. Example: -l 2 3 4')]),
3239                                           
3240               "dump": (dump, [_args_kwargs('-d', '--histograms',
3241                                     action='store_true',
3242                                     help='list histograms in files, overrides --raw'),
3243
3244                               _args_kwargs('-p', '--phases',
3245                                            action='store_true',
3246                                            help='list phases in files, overrides --raw'),
3247
3248                               _args_kwargs('-r', '--raw',
3249                                      action='store_true', help='dump raw file contents, default'),
3250
3251                               _args_kwargs('files', nargs='+')]),
3252
3253               "refine":
3254               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
3255                         _args_kwargs('refinements',
3256                                      help='JSON file of refinements to apply. if not present'
3257                                           ' refines file as-is',
3258                                      default=None,
3259                                      nargs='?')]),
3260
3261               "seqrefine": (seqrefine, []),
3262               "export": (export, [_args_kwargs('gpxfile',
3263                                                help='the project file from which to export'),
3264                                   _args_kwargs('phase', help='identifier of phase to export'),
3265                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
3266               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
3267                                                   help='list of files to browse')])}
3268
3269
3270def main():
3271    '''The command-line interface for calling GSASIIscriptable as a shell command,
3272    where it is expected to be called as::
3273
3274       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
3275
3276    The following subcommands are defined:
3277
3278        * create, see :func:`create`
3279        * add, see :func:`add`
3280        * dump, see :func:`dump`
3281        * refine, see :func:`refine`
3282        * seqrefine, see :func:`seqrefine`
3283        * export, :func:`export`
3284        * browse, see :func:`IPyBrowse`
3285
3286    .. seealso::
3287        :func:`create`
3288        :func:`add`
3289        :func:`dump`
3290        :func:`refine`
3291        :func:`seqrefine`
3292        :func:`export`
3293        :func:`IPyBrowse`
3294    '''
3295    parser = argparse.ArgumentParser(description=
3296        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
3297        )
3298    subs = parser.add_subparsers()
3299
3300    # Create all of the specified subparsers
3301    for name, (func, args) in subcommands.items():
3302        new_parser = subs.add_parser(name,help=commandhelp.get(name),
3303                                     description='Command "'+name+'" '+commandhelp.get(name))
3304        for listargs, kwds in args:
3305            new_parser.add_argument(*listargs, **kwds)
3306        new_parser.set_defaults(func=func)
3307
3308    # Parse and trigger subcommand
3309    result = parser.parse_args()
3310    result.func(result)
3311
3312if __name__ == '__main__':
3313    main()
Note: See TracBrowser for help on using the repository browser.