source: trunk/GSASIIscriptable.py @ 3811

Last change on this file since 3811 was 3811, checked in by toby, 3 years ago

scriptable: fix reading py2 gpx in py3 & use of multi-set instprm files

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 135.3 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2019-02-04 21:35:43 +0000 (Mon, 04 Feb 2019) $
5# $Author: toby $
6# $Revision: 3811 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3811 2019-02-04 21:35:43Z toby $
9########### SVN repository information ###################
10#
11# TODO: integrate 2d data based on a model
12#
13"""
14*GSASIIscriptable: Scripting Interface*
15=======================================
16
17Routines for reading, writing, modifying and creating GSAS-II project (.gpx) files.
18This file specifies several wrapper classes around GSAS-II data representations.
19They all inherit from :class:`G2ObjectWrapper`. The chief class is :class:`G2Project`,
20which represents an entire GSAS-II project and provides several methods to access
21phases, powder histograms, and execute Rietveld refinements. These routines can be
22accessed in two ways:
23the :ref:`CommandlineInterface` provides access a number of features without writing
24Python scripts of from the :ref:`API`, which allows much more versatile access from
25within Python.
26
27
28=====================
29Overview
30=====================
31The most commonly used routines are:
32
33:class:`G2Project`
34   Always needed: used to create a new project (.gpx) file or read in an existing one.
35
36:meth:`G2Project.add_powder_histogram`
37   Used to read in powder diffraction data to a project file.
38
39:meth:`G2Project.add_simulated_powder_histogram`
40   Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
41
42:meth:`G2Project.save`
43   Writes the current project to disk.
44
45:meth:`G2Project.do_refinements`
46   This is passed a list of dictionaries, where each dict defines a refinement step.
47   Passing a list with a single empty dict initiates a refinement with the current
48   parameters and flags.
49   
50:meth:`G2Project.set_refinement`
51   This is passed a single dict which is used to set parameters and flags.
52   These actions can be performed also in :meth:`G2Project.do_refinements`.
53
54Refinement dicts
55   A refinement dict sets up a single refinement step
56   (as used in :meth:`G2Project.do_refinements` and described in
57   :ref:`Project_dicts`).
58   It specifies parameter & refinement flag changes, which are usually followed by a refinement and
59   optionally by calls to locally-defined Python functions. The keys used in these dicts are
60   defined in the :ref:`Refinement_recipe` table, below.
61
62There are several ways to set parameters project using different level objects, as is
63are described in sections below, but the simplest way to access parameters and flags
64in a project is to use the above routines.
65
66=====================
67Refinement parameters
68=====================
69The most complex part of scripting GSAS-II comes in setting up the input to control refinements, which is
70described here.
71
72.. _Project_dicts:
73
74-----------------------------
75Project-level Parameter Dict
76-----------------------------
77
78As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
79which can be accessed individually by the objects that encapsulate individual phases and histograms
80but it will be usually simplest to create a composite dictionary
81that is used at the project-level. A dict is created with keys
82"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
83(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
84determine parameter values and will determine which parameters will be refined.
85
86The specific keys and subkeys that can be used are defined in tables
87:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
88
89Note that optionally a list of histograms and/or phases can be supplied in the call to
90:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
91phases and histograms.
92
93As an example:
94
95.. code-block::  python
96
97    pardict = {'set': { 'Limits': [0.8, 12.0],
98                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
99                       'Background': {'type': 'chebyschev', 'refine': True}},
100              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
101    my_project.set_refinement(pardict)
102   
103.. _Refinement_recipe:
104   
105------------------------
106Refinement recipe
107------------------------
108Building on the :ref:`Project_dicts`,
109it is possible to specify a sequence of refinement actions as a list of
110these dicts and supplying this list
111as an argument to :meth:`G2Project.do_refinements`.
112
113As an example, this code performs the same actions as in the example in the section above:
114
115.. code-block::  python
116   
117    pardict = {'set': { 'Limits': [0.8, 12.0],
118                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
119                       'Background': {'type': 'chebyschev', 'refine': True}},
120              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
121    my_project.do_refinements([pardict])
122
123However, in addition to setting a number of parameters, this example will perform a refinement as well.
124If more than one dict is specified in the list, as is done in this example,
125two refinement steps will be performed:
126
127.. code-block::  python
128
129    my_project.do_refinements([pardict,pardict1])
130
131
132The keys defined in the following table
133may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that now keys ``histograms``
134and ``phases`` are used to limit actions to specific sets of parameters within the project.
135
136========== ============================================================================
137key         explanation
138========== ============================================================================
139set                    Specifies a dict with keys and subkeys as described in the
140                       :ref:`Refinement_parameters_fmt` section. Items listed here
141                       will be set to be refined.
142clear                  Specifies a dict, as above for set, except that parameters are
143                       cleared and thus will not be refined.
144once                   Specifies a dict as above for set, except that parameters are
145                       set for the next cycle of refinement and are cleared once the
146                       refinement step is completed.
147skip                   Normally, once parameters are processed with a set/clear/once
148                       action(s), a refinement is started. If skip is defined as True
149                       (or any other value) the refinement step is not performed.
150output                 If a file name is specified for output is will be used to save
151                       the current refinement.
152histograms             Should contain a list of histogram(s) to be used for the
153                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
154                       :ref:`HAP_parameters_table`. Note that this will be
155                       ignored for :ref:`Phase_parameters_table`. Histograms may be
156                       specified as a list of strings [('PWDR ...'),...], indices
157                       [0,1,2] or as list of objects [hist1, hist2].
158phases                 Should contain a list of phase(s) to be used for the
159                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
160                       :ref:`HAP_parameters_table`. Note that this will be
161                       ignored for :ref:`Histogram_parameters_table`.
162                       Phases may be specified as a list of strings
163                       [('Phase name'),...], indices [0,1,2] or as list of objects
164                       [phase0, phase2].
165call                   Specifies a function to call after a refinement is completed.
166                       The value supplied can be the object (typically a function)
167                       that will be called or a string that will evaluate (in the
168                       namespace inside :meth:`G2Project.iter_refinements` where
169                       ``self`` references the project.)
170                       Nothing is called if this is not specified.
171callargs               Provides a list of arguments that will be passed to the function
172                       in call (if any). If call is defined and callargs is not, the
173                       current <tt>G2Project</tt> is passed as a single argument.
174========== ============================================================================
175
176An example that performs a series of refinement steps follows:
177
178.. code-block::  python
179
180    reflist = [
181            {"set": { "Limits": { "low": 0.7 },
182                      "Background": { "no. coeffs": 3,
183                                      "refine": True }}},
184            {"set": { "LeBail": True,
185                      "Cell": True }},
186            {"set": { "Sample Parameters": ["DisplaceX"]}},
187            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
188            {"set": { "Mustrain": { "type": "uniaxial",
189                                    "refine": "equatorial",
190                                    "direction": [0, 0, 1]}}},
191            {"set": { "Mustrain": { "type": "uniaxial",
192                                    "refine": "axial"}}},
193            {"clear": { "LeBail": True},
194             "set": { "Atoms": { "Mn": "X" }}},
195            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
196    my_project.do_refinements(reflist)
197   
198
199In this example, a separate refinement step will be performed for each dict in the list (since
200"skip" is not included).
201Note that in the second from last refinement step, parameters are both set and cleared.
202   
203.. _Refinement_parameters_kinds:
204
205----------------------------
206Refinement parameter types
207----------------------------
208
209Note that parameters and refinement flags used in GSAS-II fall into three classes:
210
211    * **Histogram**: There will be a set of these for each dataset loaded into a
212      project file. The parameters available depend on the type of histogram
213      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
214      include the overall scale factor, background, instrument and sample parameters;
215      see the :ref:`Histogram_parameters_table` table for a list of the histogram
216      parameters where access has been provided.
217     
218    * **Phase**: There will be a set of these for each phase loaded into a
219      project file. While some parameters are found in all types of phases,
220      others are only found in certain types (modulated, magnetic, protein...).
221      Typical phase parameters include unit cell lengths and atomic positions; see the
222      :ref:`Phase_parameters_table` table for a list of the phase     
223      parameters where access has been provided.
224     
225    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
226      that is associated with each phase, so that if there are ``N`` phases and ``M``
227      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
228      histograms are not linked to all phases.) Typical HAP parameters include the
229      phase fractions, sample microstrain and crystallite size broadening terms,
230      hydrostatic strain perturbations of the unit cell and preferred orientation
231      values.
232      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
233      been provided.
234
235.. _Refinement_parameters_fmt:
236
237=================================
238Specifying Refinement Parameters
239=================================
240
241Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
242where the details of these dicts are organized depends on the
243type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
244of keys (as described below) for each of the three types of parameters.
245
246.. _Histogram_parameters_table:
247
248--------------------
249Histogram parameters
250--------------------
251
252This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
253and :func:`G2PwdrData.clear_refinements`. As an example,
254
255.. code-block::  python
256
257   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
258                         "Sample Parameters": ["Scale"],
259                         "Limits": [10000, 40000]})
260
261With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
262``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
263key is used to define specific histograms. As an example:
264
265.. code-block::  python
266
267  gsas_proj.do_refinements([
268      {'set': {
269          'Background': {'no.coeffs': 3, 'refine': True},
270          'Sample Parameters': ['Scale'],
271          'Limits': [10000, 40000]},
272      'histograms': [1,2]}
273                            ])
274
275Note that below in the Instrument Parameters section,
276related profile parameters (such as U and V) are grouped together but
277separated by commas to save space in the table.
278
279.. tabularcolumns:: |l|l|p{3.5in}|
280
281===================== ====================  =================================================
282key                   subkey                explanation
283===================== ====================  =================================================
284Limits                                      The range of 2-theta (degrees) or TOF (in
285                                            microsec) range of values to use. Can
286                                            be either a dictionary of 'low' and/or 'high',
287                                            or a list of 2 items [low, high]
288\                     low                   Sets the low limit
289\                     high                  Sets the high limit
290
291Sample Parameters                           Should be provided as a **list** of subkeys
292                                            to set or clear, e.g. ['DisplaceX', 'Scale']
293\                     Absorption
294\                     Contrast
295\                     DisplaceX             Sample displacement along the X direction
296\                     DisplaceY             Sample displacement along the Y direction
297\                     Scale                 Histogram Scale factor
298
299Background                                  Sample background. If value is a boolean,
300                                            the background's 'refine' parameter is set
301                                            to the given boolean. Usually should be a
302                                            dictionary with any of the following keys:
303\                     type                  The background model, e.g. 'chebyschev'
304\                     refine                The value of the refine flag, boolean
305\                     no. coeffs            Number of coefficients to use, integer
306\                     coeffs                List of floats, literal values for background
307\                     FixedPoints           List of (2-theta, intensity) values for fixed points
308\                     fit fixed points      If True, triggers a fit to the fixed points to
309                                            be calculated. It is calculated when this key is
310                                            detected, regardless of calls to refine.
311
312Instrument Parameters                       As in Sample Paramters, provide as a **list** of
313                                            subkeys to
314                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
315\                     U, V, W               Gaussian peak profile terms
316\                     X, Y, Z               Lorentzian peak profile terms
317\                     alpha, beta-0,        TOF profile terms
318                      beta-1, beta-q,
319\                     sig-0, sig-1,         TOF profile terms
320                      sig-2, sig-q
321\                     difA, difB, difC      TOF Calibration constants
322\                     Zero                  Zero shift
323\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
324\                     Polariz.              Polarization parameter
325\                     Lam                   Lambda, the incident wavelength
326===================== ====================  =================================================
327
328.. _Phase_parameters_table:
329
330----------------
331Phase parameters
332----------------
333
334This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
335and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
336these parameters should be placed inside a dict with a key
337``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
338key is used to define specific phase(s).
339
340
341.. tabularcolumns:: |l|p{4.5in}|
342
343======= ==========================================================
344key                   explanation
345======= ==========================================================
346Cell                  Whether or not to refine the unit cell.
347Atoms                 Dictionary of atoms and refinement flags.
348                      Each key should be an atom label, e.g.
349                      'O3', 'Mn5', and each value should be
350                      a string defining what values to refine.
351                      Values can be any combination of 'F'
352                      for fractional occupancy, 'X' for position,
353                      and 'U' for Debye-Waller factor
354LeBail                Enables LeBail intensity extraction.
355======= ==========================================================
356
357
358.. _HAP_parameters_table:
359
360
361Histogram-and-phase parameters
362------------------------------
363
364This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
365and :func:`G2Phase.clear_HAP_refinements`. When supplied to
366:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
367``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
368unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
369
370.. tabularcolumns:: |l|l|p{3.5in}|
371
372=============  ==========  ============================================================
373key             subkey                 explanation
374=============  ==========  ============================================================
375Babinet                                Should be a **list** of the following
376                                       subkeys. If not, assumes both
377                                       BabA and BabU
378\               BabA
379\               BabU
380Extinction                             Boolean, True to refine.
381HStrain                                Boolean, True to refine all appropriate
382                                       $D_ij$ terms.
383Mustrain
384\               type                   Mustrain model. One of 'isotropic',
385                                       'uniaxial', or 'generalized'. Should always
386                                       be specified.
387\              direction               For uniaxial only. A list of three
388                                       integers,
389                                       the [hkl] direction of the axis.
390\               refine                 Usually boolean, set to True to refine.
391                                       or False to clear.
392                                       For uniaxial model, can specify a value
393                                       of 'axial' or 'equatorial' to set that flag
394                                       to True or a single
395                                       boolean sets both axial and equatorial.
396Size                                   
397\               type                   Size broadening model. One of 'isotropic',
398                                       'uniaxial', or 'ellipsoid'. Should always
399                                       be specified.
400\              direction               For uniaxial only. A list of three
401                                       integers,
402                                       the [hkl] direction of the axis.
403\               refine                 Boolean, True to refine.
404\               value                  float, size value in microns
405Pref.Ori.                              Boolean, True to refine
406Show                                   Boolean, True to refine
407Use                                    Boolean, True to refine
408Scale                                  Phase fraction; Boolean, True to refine
409=============  ==========  ============================================================
410
411------------------------
412Histogram/Phase objects
413------------------------
414Each phase and powder histogram in a :class:`G2Project` object has an associated
415object. Parameters within each individual object can be turned on and off by calling
416:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
417for histogram parameters;
418:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
419for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
420:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
421
422.. code-block::  python
423
424    params = { 'Limits': [0.8, 12.0],
425               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
426               'Background': {'type': 'chebyschev', 'refine': True}}
427    some_histogram.set_refinements(params)
428
429Likewise to turn refinement flags on, use code such as this:
430
431.. code-block::  python
432
433    params = { 'Instrument Parameters': ['U', 'V', 'W']}
434    some_histogram.set_refinements(params)
435
436and to turn these refinement flags, off use this (Note that the
437``.clear_refinements()`` methods will usually will turn off refinement even
438if a refinement parameter is set in the dict to True.):
439
440.. code-block::  python
441
442    params = { 'Instrument Parameters': ['U', 'V', 'W']}
443    some_histogram.clear_refinements(params)
444
445For phase parameters, use code such as this:
446   
447.. code-block::  python
448
449    params = { 'LeBail': True, 'Cell': True,
450               'Atoms': { 'Mn1': 'X',
451                          'O3': 'XU',
452                          'V4': 'FXU'}}
453    some_histogram.set_refinements(params)
454
455
456and here is an example for HAP parameters:
457
458.. code-block::  python
459
460    params = { 'Babinet': 'BabA',
461               'Extinction': True,
462               'Mustrain': { 'type': 'uniaxial',
463                             'direction': [0, 0, 1],
464                             'refine': True}}
465    some_phase.set_HAP_refinements(params)
466
467Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
468
469.. _CommandlineInterface:
470
471=======================================
472GSASIIscriptable Command-line Interface
473=======================================
474
475The routines described above are intended to be called from a Python script, but an
476alternate way to access some of the same functionality is to
477invoke the ``GSASIIscriptable.py`` script from
478the command line usually from within a shell script or batch file. This
479will usually be done with a command such as::
480
481       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
482
483    The following subcommands are defined:
484
485        * create, see :func:`create`
486        * add, see :func:`add`
487        * dump, see :func:`dump`
488        * refine, see :func:`refine`
489        * seqrefine, see :func:`seqrefine`
490        * export, :func:`export`
491        * browse, see :func:`IPyBrowse`
492       
493Run::
494
495   python GSASIIscriptable.py --help
496
497to show the available subcommands, and inspect each subcommand with
498`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
499
500.. _JsonFormat:
501
502-------------------------
503Parameters in JSON files
504-------------------------
505
506The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
507a JSON format file
508(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
509This dict may have two keys:
510
511refinements:
512  This defines the a set of refinement steps in a JSON representation of a
513  :ref:`Refinement_recipe` list.
514
515code:
516  This optionally defines Python code that will be executed after the project is loaded,
517  but before the refinement is started. This can be used to execute Python code to change
518  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
519  project object is accessed with variable ``proj``) or to define code that will be called
520  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
521   
522JSON website: `Introducing JSON <http://json.org/>`_.
523
524.. _API:
525
526============================================================
527GSASIIscriptable Application Layer (API)
528============================================================
529
530The large number of classes and modules in this module are described below.
531Most commonly a script will create a G2Project object using :class:`G2Project` and then
532perform actions such as
533adding a histogram (method :meth:`G2Project.add_powder_histogram`),
534adding a phase (method :meth:`G2Project.add_phase`),
535or setting parameters and performing a refinement
536(method :meth:`G2Project.do_refinements`).
537
538In some cases, it may be easier to use
539methods inside :class:`G2PwdrData` or :class:`G2Phase` or these objects
540may offer more options.
541
542---------------------------------------------------------------
543Complete Documentation: All classes and functions
544---------------------------------------------------------------
545"""
546from __future__ import division, print_function
547import argparse
548import os.path as ospath
549import datetime as dt
550import sys
551import platform
552if '2' in platform.python_version_tuple()[0]:
553    import cPickle
554    strtypes = (str,unicode)
555else:
556    import _pickle as cPickle
557    strtypes = (str,bytes)
558import imp
559import copy
560import os
561import random as ran
562
563import numpy.ma as ma
564import scipy.interpolate as si
565import numpy as np
566import scipy as sp
567
568import GSASIIpath
569GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
570import GSASIIobj as G2obj
571import GSASIIpwd as G2pwd
572import GSASIIstrMain as G2strMain
573import GSASIIstrIO as G2strIO
574import GSASIIspc as G2spc
575import GSASIIElem as G2elem
576
577
578# Delay imports to not slow down small scripts
579G2fil = None
580PwdrDataReaders = []
581PhaseReaders = []
582exportersByExtension = {}
583'''Specifies the list of extensions that are supported for Powder data export'''
584
585def LoadG2fil():
586    """Delay importing this module, it is slow"""
587    global G2fil
588    if G2fil is None:
589        import GSASIIfiles
590        G2fil = GSASIIfiles
591        global PwdrDataReaders
592        global PhaseReaders
593        PwdrDataReaders = G2fil.LoadImportRoutines("pwd", "Powder_Data")
594        PhaseReaders = G2fil.LoadImportRoutines("phase", "Phase")
595        AllExporters = G2fil.LoadExportRoutines(None)
596        global exportersByExtension
597        exportersByExtension = {}
598        for obj in AllExporters:
599            try:
600                obj.Writer
601            except AttributeError:
602                continue
603            for typ in obj.exporttype:
604                if typ not in exportersByExtension:
605                    exportersByExtension[typ] = {obj.extension:obj}
606                else:
607                    exportersByExtension[typ][obj.extension] = obj
608
609def LoadDictFromProjFile(ProjFile):
610    '''Read a GSAS-II project file and load items to dictionary
611   
612    :param str ProjFile: GSAS-II project (name.gpx) full file name
613    :returns: Project,nameList, where
614
615      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
616        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
617        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
618      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
619
620    Example for fap.gpx::
621
622      Project = {                 #NB:dict order is not tree order
623        'Phases':{'data':None,'fap':{phase dict}},
624        'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
625        'Rigid bodies':{'data': {rigid body dict}},
626        'Covariance':{'data':{covariance data dict}},
627        'Controls':{'data':{controls data dict}},
628        'Notebook':{'data':[notebook list]},
629        'Restraints':{'data':{restraint data dict}},
630        'Constraints':{'data':{constraint data dict}}]
631        }
632      nameList = [                #NB: reproduces tree order
633        ['Notebook',],
634        ['Controls',],
635        ['Covariance',],
636        ['Constraints',],
637        ['Restraints',],
638        ['Rigid bodies',],
639        ['PWDR FAP.XRA Bank 1',
640             'Comments',
641             'Limits',
642             'Background',
643             'Instrument Parameters',
644             'Sample Parameters',
645             'Peak List',
646             'Index Peak List',
647             'Unit Cells List',
648             'Reflection Lists'],
649        ['Phases', 'fap']
650        ]
651    '''
652    # Let IOError be thrown if file does not exist
653    if not ospath.exists(ProjFile):
654        print ('\n*** Error attempt to open project file that does not exist: \n    {}'.
655                   format(ProjFile))
656        raise IOError('GPX file {} does not exist'.format(ProjFile))
657    try:
658        Project, nameList = G2strIO.GetFullGPX(ProjFile)
659    except Exception as msg:
660        raise IOError(msg)
661    return Project,nameList
662
663def SaveDictToProjFile(Project,nameList,ProjFile):
664    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
665
666    :param dict Project: representation of gpx file following the GSAS-II
667        tree structure as described for LoadDictFromProjFile
668    :param list nameList: names of main tree entries & subentries used to reconstruct project file
669    :param str ProjFile: full file name for output project.gpx file (including extension)
670    '''
671    file = open(ProjFile,'wb')
672    try:
673        for name in nameList:
674            data = []
675            item = Project[name[0]]
676            data.append([name[0],item['data']])
677            for item2 in name[1:]:
678                data.append([item2,item[item2]])
679            cPickle.dump(data,file,1)
680    finally:
681        file.close()
682
683# def ImportPowder(reader,filename):
684#     '''Use a reader to import a powder diffraction data file
685
686#     :param str reader: a scriptable reader
687#     :param str filename: full name of powder data file; can be "multi-Bank" data
688
689#     :returns: list rdlist: list of reader objects containing powder data, one for each
690#         "Bank" of data encountered in file. Items in reader object of interest are:
691
692#           * rd.comments: list of str: comments found on powder file
693#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
694#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
695#     '''
696#     rdfile,rdpath,descr = imp.find_module(reader)
697#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
698#     rd = rdclass.GSAS_ReaderClass()
699#     if not rd.scriptable:
700#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
701#         return None
702#     rdlist = []
703#     if rd.ContentsValidator(filename):
704#         repeat = True
705#         rdbuffer = {} # create temporary storage for file reader
706#         block = 0
707#         while repeat: # loop if the reader asks for another pass on the file
708#             block += 1
709#             repeat = False
710#             rd.objname = ospath.basename(filename)
711#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
712#             if flag:
713#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
714#                 if rd.repeat:
715#                     repeat = True
716#         return rdlist
717#     print(rd.errors)
718#     return None
719
720def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
721    '''Create an initial Histogram dictionary
722
723    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
724    '''
725    if dType in ['SXC','SNC']:
726        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
727            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
728            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
729            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
730            'Flack':[0.0,False]}
731    elif dType == 'SNT':
732        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
733            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
734            'Extinction':['Lorentzian','None', {
735            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
736    elif 'P' in dType:
737        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
738            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
739            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
740                [1.,1.,1.,0.,0.,0.],6*[False,]],
741            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
742                NShkl*[0.01,],NShkl*[False,]],
743            'HStrain':[NDij*[0.0,],NDij*[False,]],
744            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
745
746
747def PreSetup(data):
748    '''Create part of an initial (empty) phase dictionary
749
750    from GSASIIphsGUI.py, near end of UpdatePhaseData
751
752    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
753    '''
754    if 'RBModels' not in data:
755        data['RBModels'] = {}
756    if 'MCSA' not in data:
757        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
758    if 'dict' in str(type(data['MCSA']['Results'])):
759        data['MCSA']['Results'] = []
760    if 'Modulated' not in data['General']:
761        data['General']['Modulated'] = False
762#    if 'modulated' in data['General']['Type']:
763#        data['General']['Modulated'] = True
764#        data['General']['Type'] = 'nuclear'
765
766
767def SetupGeneral(data, dirname):
768    """Helps initialize phase data.
769
770    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
771
772    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
773    """
774    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
775                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
776    generalData = data['General']
777    atomData = data['Atoms']
778    generalData['AtomTypes'] = []
779    generalData['Isotopes'] = {}
780
781    if 'Isotope' not in generalData:
782        generalData['Isotope'] = {}
783    if 'Data plot type' not in generalData:
784        generalData['Data plot type'] = 'Mustrain'
785    if 'POhkl' not in generalData:
786        generalData['POhkl'] = [0,0,1]
787    if 'Map' not in generalData:
788        generalData['Map'] = mapDefault.copy()
789    if 'Flip' not in generalData:
790        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
791            'k-factor':0.1,'k-Max':20.,}
792    if 'testHKL' not in generalData['Flip']:
793        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
794    if 'doPawley' not in generalData:
795        generalData['doPawley'] = False     #ToDo: change to ''
796    if 'Pawley dmin' not in generalData:
797        generalData['Pawley dmin'] = 1.0
798    if 'Pawley neg wt' not in generalData:
799        generalData['Pawley neg wt'] = 0.0
800    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
801        'MCSA controls' not in generalData:
802        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
803        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
804        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
805    if 'AtomPtrs' not in generalData:
806        generalData['AtomPtrs'] = [3,1,7,9]
807        if generalData['Type'] == 'macromolecular':
808            generalData['AtomPtrs'] = [6,4,10,12]
809        elif generalData['Type'] == 'magnetic':
810            generalData['AtomPtrs'] = [3,1,10,12]
811    if generalData['Modulated']:
812        generalData['Type'] = 'nuclear'
813        if 'Super' not in generalData:
814            generalData['Super'] = 1
815            generalData['SuperVec'] = [[0,0,.1],False,4]
816            generalData['SSGData'] = {}
817        if '4DmapData' not in generalData:
818            generalData['4DmapData'] = mapDefault.copy()
819            generalData['4DmapData'].update({'MapType':'Fobs'})
820    if 'Modulated' not in generalData:
821        generalData['Modulated'] = False
822    if 'HydIds' not in generalData:
823        generalData['HydIds'] = {}
824    cx,ct,cs,cia = generalData['AtomPtrs']
825    generalData['NoAtoms'] = {}
826    generalData['BondRadii'] = []
827    generalData['AngleRadii'] = []
828    generalData['vdWRadii'] = []
829    generalData['AtomMass'] = []
830    generalData['Color'] = []
831    if generalData['Type'] == 'magnetic':
832        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
833        landeg = generalData.get('Lande g',[])
834    generalData['Mydir'] = dirname
835    badList = {}
836    for iat,atom in enumerate(atomData):
837        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
838        if generalData['AtomTypes'].count(atom[ct]):
839            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
840        elif atom[ct] != 'UNK':
841            Info = G2elem.GetAtomInfo(atom[ct])
842            if not Info:
843                if atom[ct] not in badList:
844                    badList[atom[ct]] = 0
845                badList[atom[ct]] += 1
846                atom[ct] = 'UNK'
847                continue
848            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
849            generalData['AtomTypes'].append(atom[ct])
850            generalData['Z'] = Info['Z']
851            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
852            generalData['BondRadii'].append(Info['Drad'])
853            generalData['AngleRadii'].append(Info['Arad'])
854            generalData['vdWRadii'].append(Info['Vdrad'])
855            if atom[ct] in generalData['Isotope']:
856                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
857                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
858                    generalData['Isotope'][atom[ct]] = isotope
859                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
860            else:
861                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
862                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
863                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
864                    generalData['Isotope'][atom[ct]] = isotope
865                generalData['AtomMass'].append(Info['Mass'])
866            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
867            generalData['Color'].append(Info['Color'])
868            if generalData['Type'] == 'magnetic':
869                if len(landeg) < len(generalData['AtomTypes']):
870                    landeg.append(2.0)
871    if generalData['Type'] == 'magnetic':
872        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
873
874    if badList:
875        msg = 'Warning: element symbol(s) not found:'
876        for key in badList:
877            msg += '\n\t' + key
878            if badList[key] > 1:
879                msg += ' (' + str(badList[key]) + ' times)'
880        raise G2ScriptException("Phase error:\n" + msg)
881        # wx.MessageBox(msg,caption='Element symbol error')
882    F000X = 0.
883    F000N = 0.
884    for i,elem in enumerate(generalData['AtomTypes']):
885        F000X += generalData['NoAtoms'][elem]*generalData['Z']
886        isotope = generalData['Isotope'][elem]
887        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
888    generalData['F000X'] = F000X
889    generalData['F000N'] = F000N
890    import GSASIImath as G2mth
891    generalData['Mass'] = G2mth.getMass(generalData)
892
893
894def make_empty_project(author=None, filename=None):
895    """Creates an dictionary in the style of GSASIIscriptable, for an empty
896    project.
897
898    If no author name or filename is supplied, 'no name' and
899    <current dir>/test_output.gpx are used , respectively.
900
901    Returns: project dictionary, name list
902
903    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
904    """
905    if not filename:
906        filename = 'test_output.gpx'
907    filename = os.path.abspath(filename)
908    gsasii_version = str(GSASIIpath.GetVersionNumber())
909    LoadG2fil()
910    import matplotlib as mpl
911    python_library_versions = G2fil.get_python_versions([mpl, np, sp])
912
913    controls_data = dict(G2obj.DefaultControls)
914    controls_data['LastSavedAs'] = filename
915    controls_data['LastSavedUsing'] = gsasii_version
916    controls_data['PythonVersions'] = python_library_versions
917    if author:
918        controls_data['Author'] = author
919
920    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
921                                       'Global': []}},
922              'Controls': {'data': controls_data},
923              u'Covariance': {'data': {}},
924              u'Notebook': {'data': ['']},
925              u'Restraints': {'data': {}},
926              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
927                                'Residue': {'AtInfo': {}},
928                                'Vector':  {'AtInfo': {}}}}}
929
930    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
931             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
932
933    return output, names
934
935
936class G2ImportException(Exception):
937    pass
938
939class G2ScriptException(Exception):
940    pass
941
942def import_generic(filename, readerlist, fmthint=None, bank=None):
943    """Attempt to import a filename, using a list of reader objects.
944
945    Returns the first reader object which worked."""
946    # Translated from OnImportGeneric method in GSASII.py
947    primaryReaders, secondaryReaders = [], []
948    for reader in readerlist:
949        if fmthint is not None and fmthint not in reader.formatName: continue
950        flag = reader.ExtensionValidator(filename)
951        if flag is None:
952            secondaryReaders.append(reader)
953        elif flag:
954            primaryReaders.append(reader)
955    if not secondaryReaders and not primaryReaders:
956        raise G2ImportException("Could not read file: ", filename)
957
958    with open(filename, 'Ur') as fp:
959        rd_list = []
960
961        for rd in primaryReaders + secondaryReaders:
962            # Initialize reader
963            rd.selections = []
964            if bank is None:
965                rd.selections = []
966            else:
967                rd.selections = [bank-1]
968            rd.dnames = []
969            rd.ReInitialize()
970            # Rewind file
971            rd.errors = ""
972            if not rd.ContentsValidator(filename):
973                # Report error
974                print("Warning: File {} has a validation error, continuing".format(filename))
975            if len(rd.selections) > 1:
976                raise G2ImportException("File {} has {} banks. Specify which bank to read with databank param."
977                                .format(filename,len(rd.selections)))
978
979            block = 0
980            rdbuffer = {}
981            repeat = True
982            while repeat:
983                repeat = False
984                block += 1
985                rd.objname = os.path.basename(filename)
986                try:
987                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
988                except:
989                    flag = False
990                if flag:
991                    # Omitting image loading special cases
992                    rd.readfilename = filename
993                    rd_list.append(copy.deepcopy(rd))
994                    repeat = rd.repeat
995                else:
996                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
997            if rd_list:
998                if rd.warnings:
999                    print("Read warning by", rd.formatName, "reader:",
1000                          rd.warnings, file=sys.stderr)
1001                elif bank is None:
1002                    print("{} read by Reader {}"
1003                              .format(filename,rd.formatName))
1004                else:
1005                    print("{} block # {} read by Reader {}"
1006                              .format(filename,bank,rd.formatName))
1007                return rd_list
1008    raise G2ImportException("No reader could read file: " + filename)
1009
1010
1011def load_iprms(instfile, reader, bank=None):
1012    """Loads instrument parameters from a file, and edits the
1013    given reader.
1014
1015    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1016    """
1017    LoadG2fil()
1018    ext = os.path.splitext(instfile)[1]
1019
1020    if ext.lower() == '.instprm':
1021        # New GSAS File, load appropriate bank
1022        with open(instfile) as f:
1023            lines = f.readlines()
1024        if bank is None: 
1025            bank = reader.powderentry[2] 
1026        numbanks = reader.numbanks
1027        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1028        reader.instfile = instfile
1029        reader.instmsg = '{} (G2 fmt) bank {}'.format(instfile,bank)
1030        return iparms
1031    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1032        raise ValueError('Expected .prm file, found: ', instfile)
1033
1034    # It's an old GSAS file, load appropriately
1035    Iparm = {}
1036    with open(instfile, 'Ur') as fp:
1037        for line in fp:
1038            if '#' in line:
1039                continue
1040            Iparm[line[:12]] = line[12:-1]
1041    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1042    if bank is not None:
1043        # pull out requested bank # bank from the data, and change the bank to 1
1044        Iparm,IparmC = {},Iparm
1045        for key in IparmC:
1046            if 'INS' not in key[:3]: continue   #skip around rubbish lines in some old iparm
1047            if key[4:6] == "  ":
1048                Iparm[key] = IparmC[key]
1049            elif int(key[4:6].strip()) == bank:
1050                Iparm[key[:4]+' 1'+key[6:]] = IparmC[key]           
1051        reader.instbank = bank
1052    elif ibanks == 1:
1053        reader.instbank = 1
1054    else: 
1055        raise G2ImportException("Instrument parameter file has {} banks, select one with instbank param."
1056                                    .format(ibanks))
1057    reader.powderentry[2] = 1
1058    reader.instfile = instfile
1059    reader.instmsg = '{} bank {}'.format(instfile,reader.instbank)
1060    return G2fil.SetPowderInstParms(Iparm, reader)
1061
1062def load_pwd_from_reader(reader, instprm, existingnames=[],bank=None):
1063    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1064
1065    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1066
1067    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1068    """
1069    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1070    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1071
1072    try:
1073        Iparm1, Iparm2 = instprm
1074    except ValueError:
1075        Iparm1, Iparm2 = load_iprms(instprm, reader, bank=bank)
1076        print('Instrument parameters read:',reader.instmsg)
1077    Ymin = np.min(reader.powderdata[1])
1078    Ymax = np.max(reader.powderdata[1])
1079    valuesdict = {'wtFactor': 1.0,
1080                  'Dummy': False,
1081                  'ranId': ran.randint(0, sys.maxsize),
1082                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1083                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1084                  'Yminmax': [Ymin, Ymax]}
1085    reader.Sample['ranId'] = valuesdict['ranId']
1086
1087    # Ending keys:
1088    # [u'Reflection Lists',
1089    #  u'Limits',
1090    #  'data',
1091    #  u'Index Peak List',
1092    #  u'Comments',
1093    #  u'Unit Cells List',
1094    #  u'Sample Parameters',
1095    #  u'Peak List',
1096    #  u'Background',
1097    #  u'Instrument Parameters']
1098    Tmin = np.min(reader.powderdata[0])
1099    Tmax = np.max(reader.powderdata[0])
1100
1101    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1102                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1103
1104    output_dict = {u'Reflection Lists': {},
1105                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1106                   u'data': [valuesdict, reader.powderdata, HistName],
1107                   u'Index Peak List': [[], []],
1108                   u'Comments': reader.comments,
1109                   u'Unit Cells List': [],
1110                   u'Sample Parameters': reader.Sample,
1111                   u'Peak List': {'peaks': [], 'sigDict': {}},
1112                   u'Background': reader.pwdparms.get('Background', default_background),
1113                   u'Instrument Parameters': [Iparm1, Iparm2],
1114                   }
1115
1116    names = [u'Comments',
1117             u'Limits',
1118             u'Background',
1119             u'Instrument Parameters',
1120             u'Sample Parameters',
1121             u'Peak List',
1122             u'Index Peak List',
1123             u'Unit Cells List',
1124             u'Reflection Lists']
1125
1126    # TODO controls?? GSASII.py:1664-7
1127
1128    return HistName, [HistName] + names, output_dict
1129
1130
1131def _deep_copy_into(from_, into):
1132    """Helper function for reloading .gpx file. See G2Project.reload()
1133
1134    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1135    """
1136    if isinstance(from_, dict) and isinstance(into, dict):
1137        combined_keys = set(from_.keys()).union(into.keys())
1138        for key in combined_keys:
1139            if key in from_ and key in into:
1140                both_dicts = (isinstance(from_[key], dict)
1141                              and isinstance(into[key], dict))
1142                both_lists = (isinstance(from_[key], list)
1143                              and isinstance(into[key], list))
1144                if both_dicts or both_lists:
1145                    _deep_copy_into(from_[key], into[key])
1146                else:
1147                    into[key] = from_[key]
1148            elif key in from_:
1149                into[key] = from_[key]
1150            else:  # key in into
1151                del into[key]
1152    elif isinstance(from_, list) and isinstance(into, list):
1153        if len(from_) == len(into):
1154            for i in range(len(from_)):
1155                both_dicts = (isinstance(from_[i], dict)
1156                              and isinstance(into[i], dict))
1157                both_lists = (isinstance(from_[i], list)
1158                              and isinstance(into[i], list))
1159                if both_dicts or both_lists:
1160                    _deep_copy_into(from_[i], into[i])
1161                else:
1162                    into[i] = from_[i]
1163        else:
1164            into[:] = from_
1165
1166
1167class G2ObjectWrapper(object):
1168    """Base class for all GSAS-II object wrappers.
1169
1170    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1171    of overrides are implemented so that the wrapper behaves like a dictionary.
1172
1173    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1174    """
1175    def __init__(self, datadict):
1176        self.data = datadict
1177
1178    def __getitem__(self, key):
1179        return self.data[key]
1180
1181    def __setitem__(self, key, value):
1182        self.data[key] = value
1183
1184    def __contains__(self, key):
1185        return key in self.data
1186
1187    def get(self, k, d=None):
1188        return self.data.get(k, d)
1189
1190    def keys(self):
1191        return self.data.keys()
1192
1193    def values(self):
1194        return self.data.values()
1195
1196    def items(self):
1197        return self.data.items()
1198
1199
1200class G2Project(G2ObjectWrapper):
1201    """
1202    Represents an entire GSAS-II project.
1203
1204    :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1205            creates an empty project.
1206    :param str author: Author's name (not yet implemented)
1207    :param str newgpx: The filename the project should be saved to in
1208            the future. If both newgpx and gpxfile are present, the project is
1209            loaded from the gpxfile, then when saved will be written to newgpx.
1210    :param str filename: Name to be used to save the project. Has same function as
1211            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1212            is preferred over filename.
1213
1214    There are two ways to initialize this object:
1215
1216    >>> # Load an existing project file
1217    >>> proj = G2Project('filename.gpx')
1218   
1219    >>> # Create a new project
1220    >>> proj = G2Project(newgpx='new_file.gpx')
1221   
1222    Histograms can be accessed easily.
1223
1224    >>> # By name
1225    >>> hist = proj.histogram('PWDR my-histogram-name')
1226   
1227    >>> # Or by index
1228    >>> hist = proj.histogram(0)
1229    >>> assert hist.id == 0
1230   
1231    >>> # Or by random id
1232    >>> assert hist == proj.histogram(hist.ranId)
1233
1234    Phases can be accessed the same way.
1235
1236    >>> phase = proj.phase('name of phase')
1237
1238    New data can also be loaded via :meth:`~G2Project.add_phase` and
1239    :meth:`~G2Project.add_powder_histogram`.
1240
1241    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1242                                         'instrument_parameters.prm')
1243    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1244
1245    Parameters for Rietveld refinement can be turned on and off as well.
1246    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1247    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1248    """
1249    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1250        if filename is not None and newgpx is not None:
1251            raise G2ScriptException('Do not use filename and newgpx together')
1252        elif newgpx is not None:
1253            filename = newgpx
1254        if gpxfile is None:
1255            filename = os.path.abspath(os.path.expanduser(filename))
1256            self.filename = filename
1257            self.data, self.names = make_empty_project(author=author, filename=filename)
1258        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1259                                       # filename is valid, etc.
1260            if isinstance(filename, str): 
1261                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1262            else: 
1263                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1264            # TODO set author
1265            self.data, self.names = LoadDictFromProjFile(gpxfile)
1266            self.update_ids()
1267        else:
1268            raise ValueError("Not sure what to do with gpxfile")
1269
1270    @classmethod
1271    def from_dict_and_names(cls, gpxdict, names, filename=None):
1272        """Creates a :class:`G2Project` directly from
1273        a dictionary and a list of names. If in doubt, do not use this.
1274
1275        :returns: a :class:`G2Project`
1276        """
1277        out = cls()
1278        if filename:
1279            filename = os.path.abspath(os.path.expanduser(filename))
1280            out.filename = filename
1281            gpxdict['Controls']['data']['LastSavedAs'] = filename
1282        else:
1283            try:
1284                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1285            except KeyError:
1286                out.filename = None
1287        out.data = gpxdict
1288        out.names = names
1289
1290    def save(self, filename=None):
1291        """Saves the project, either to the current filename, or to a new file.
1292
1293        Updates self.filename if a new filename provided"""
1294        # TODO update LastSavedUsing ?
1295        if filename:
1296            filename = os.path.abspath(os.path.expanduser(filename))
1297            self.data['Controls']['data']['LastSavedAs'] = filename
1298            self.filename = filename
1299        elif not self.filename:
1300            raise AttributeError("No file name to save to")
1301        SaveDictToProjFile(self.data, self.names, self.filename)
1302
1303    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None,
1304                                 databank=None, instbank=None):
1305        """Loads a powder data histogram into the project.
1306
1307        Automatically checks for an instrument parameter file, or one can be
1308        provided. Note that in unix fashion, "~" can be used to indicate the
1309        home directory (e.g. ~/G2data/data.fxye).
1310
1311        :param str datafile: The powder data file to read, a filename.
1312        :param str iparams: The instrument parameters file, a filename.
1313        :param list phases: Phases to link to the new histogram
1314        :param str fmthint: If specified, only importers where the format name
1315          (reader.formatName, as shown in Import menu) contains the
1316          supplied string will be tried as importers. If not specified, all
1317          importers consistent with the file extension will be tried
1318          (equivalent to "guess format" in menu).
1319        :param int databank: Specifies a dataset number to read, if file contains
1320          more than set of data. This should be 1 to read the first bank in
1321          the file (etc.) regardless of the number on the Bank line, etc.
1322          Default is None which means there should only be one dataset in the
1323          file.
1324        :param int instbank: Specifies an instrument parameter set to read, if
1325          the instrument parameter file contains more than set of parameters.
1326          This will match the INS # in an GSAS type file so it will typically
1327          be 1 to read the first parameter set in the file (etc.)
1328          Default is None which means there should only be one parameter set
1329          in the file.
1330
1331        :returns: A :class:`G2PwdrData` object representing
1332            the histogram
1333        """
1334        LoadG2fil()
1335        datafile = os.path.abspath(os.path.expanduser(datafile))
1336        iparams = os.path.abspath(os.path.expanduser(iparams))
1337        pwdrreaders = import_generic(datafile, PwdrDataReaders,fmthint=fmthint,bank=databank)
1338        histname, new_names, pwdrdata = load_pwd_from_reader(
1339                                          pwdrreaders[0], iparams,
1340                                          [h.name for h in self.histograms()],bank=instbank)
1341        if histname in self.data:
1342            print("Warning - redefining histogram", histname)
1343        elif self.names[-1][0] == 'Phases':
1344            self.names.insert(-1, new_names)
1345        else:
1346            self.names.append(new_names)
1347        self.data[histname] = pwdrdata
1348        self.update_ids()
1349
1350        for phase in phases:
1351            phase = self.phase(phase)
1352            self.link_histogram_phase(histname, phase)
1353
1354        return self.histogram(histname)
1355
1356    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1357                                       wavelength=None, scale=None, phases=[]):
1358        """Loads a powder data histogram into the project.
1359
1360        Requires an instrument parameter file.
1361        Note that in unix fashion, "~" can be used to indicate the
1362        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1363        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1364        as the instrument type.
1365
1366        :param str histname: A name for the histogram to be created.
1367        :param str iparams: The instrument parameters file, a filename.
1368        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1369        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1370        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1371        :param float wavelength: Wavelength for CW instruments, overriding the value
1372           in the instrument parameters file if specified.
1373        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1374           simulated noise is added to the pattern, so that if the maximum intensity is
1375           small, the noise will mask the computed pattern. The scale
1376           needs to be a large number for CW neutrons.
1377           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1378        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1379           all defined phases.
1380
1381        :returns: A :class:`G2PwdrData` object representing the histogram
1382        """
1383        LoadG2fil()
1384        iparams = os.path.abspath(os.path.expanduser(iparams))
1385        if not os.path.exists(iparams):
1386            raise G2ScriptException("File does not exist:"+iparams)
1387        rd = G2obj.ImportPowderData( # Initialize a base class reader
1388            extensionlist=tuple(),
1389            strictExtension=False,
1390            formatName = 'Simulate dataset',
1391            longFormatName = 'Compute a simulated pattern')
1392        rd.powderentry[0] = '' # no filename
1393        rd.powderentry[2] = 1 # only one bank
1394        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1395        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1396        if Tmax < Tmin:
1397            Tmin,Tmax = Tmax,Tmin
1398        Tstep = abs(Tstep)
1399        if 'TOF' in rd.idstring:
1400                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1401                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1402                N = len(x)
1403        else:           
1404                N = int((Tmax-Tmin)/Tstep)+1
1405                x = np.linspace(Tmin,Tmax,N,True)
1406                N = len(x)
1407        if N < 3:
1408            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1409        rd.powderdata = [
1410            np.array(x), # x-axis values
1411            np.zeros_like(x), # powder pattern intensities
1412            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1413            np.zeros_like(x), # calc. intensities (zero)
1414            np.zeros_like(x), # calc. background (zero)
1415            np.zeros_like(x), # obs-calc profiles
1416            ]
1417        Tmin = rd.powderdata[0][0]
1418        Tmax = rd.powderdata[0][-1]
1419        rd.idstring = histname
1420        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1421                                                            [h.name for h in self.histograms()])
1422        if histname in self.data:
1423            print("Warning - redefining histogram", histname)
1424        elif self.names[-1][0] == 'Phases':
1425            self.names.insert(-1, new_names)
1426        else:
1427            self.names.append(new_names)
1428        if scale is not None:
1429            pwdrdata['Sample Parameters']['Scale'][0] = scale
1430        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1431            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1432        self.data[histname] = pwdrdata
1433        self.update_ids()
1434
1435        for phase in phases:
1436            phase = self.phase(phase)
1437            self.link_histogram_phase(histname, phase)
1438
1439        return self.histogram(histname)
1440   
1441    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1442        """Loads a phase into the project from a .cif file
1443
1444        :param str phasefile: The CIF file from which to import the phase.
1445        :param str phasename: The name of the new phase, or None for the default
1446        :param list histograms: The names of the histograms to associate with
1447            this phase. Use proj.Histograms() to add to all histograms.
1448        :param str fmthint: If specified, only importers where the format name
1449          (reader.formatName, as shown in Import menu) contains the
1450          supplied string will be tried as importers. If not specified, all
1451          importers consistent with the file extension will be tried
1452          (equivalent to "guess format" in menu).
1453
1454        :returns: A :class:`G2Phase` object representing the
1455            new phase.
1456        """
1457        LoadG2fil()
1458        histograms = [self.histogram(h).name for h in histograms]
1459        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1460
1461        # TODO handle multiple phases in a file
1462        phasereaders = import_generic(phasefile, PhaseReaders, fmthint=fmthint)
1463        phasereader = phasereaders[0]
1464       
1465        phasename = phasename or phasereader.Phase['General']['Name']
1466        phaseNameList = [p.name for p in self.phases()]
1467        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1468        phasereader.Phase['General']['Name'] = phasename
1469
1470        if 'Phases' not in self.data:
1471            self.data[u'Phases'] = { 'data': None }
1472        assert phasename not in self.data['Phases'], "phase names should be unique"
1473        self.data['Phases'][phasename] = phasereader.Phase
1474
1475        if phasereader.Constraints:
1476            Constraints = self.data['Constraints']
1477            for i in phasereader.Constraints:
1478                if isinstance(i, dict):
1479                    if '_Explain' not in Constraints:
1480                        Constraints['_Explain'] = {}
1481                    Constraints['_Explain'].update(i)
1482                else:
1483                    Constraints['Phase'].append(i)
1484
1485        data = self.data['Phases'][phasename]
1486        generalData = data['General']
1487        SGData = generalData['SGData']
1488        NShkl = len(G2spc.MustrainNames(SGData))
1489        NDij = len(G2spc.HStrainNames(SGData))
1490        Super = generalData.get('Super', 0)
1491        if Super:
1492            SuperVec = np.array(generalData['SuperVec'][0])
1493        else:
1494            SuperVec = []
1495        UseList = data['Histograms']
1496
1497        for hist in histograms:
1498            self.link_histogram_phase(hist, phasename)
1499
1500        for obj in self.names:
1501            if obj[0] == 'Phases':
1502                phasenames = obj
1503                break
1504        else:
1505            phasenames = [u'Phases']
1506            self.names.append(phasenames)
1507        phasenames.append(phasename)
1508
1509        # TODO should it be self.filename, not phasefile?
1510        SetupGeneral(data, os.path.dirname(phasefile))
1511        self.index_ids()
1512
1513        self.update_ids()
1514        return self.phase(phasename)
1515
1516    def link_histogram_phase(self, histogram, phase):
1517        """Associates a given histogram and phase.
1518
1519        .. seealso::
1520
1521            :meth:`G2Project.histogram`
1522            :meth:`G2Project.phase`"""
1523        hist = self.histogram(histogram)
1524        phase = self.phase(phase)
1525
1526        generalData = phase['General']
1527
1528        if hist.name.startswith('HKLF '):
1529            raise NotImplementedError("HKLF not yet supported")
1530        elif hist.name.startswith('PWDR '):
1531            hist['Reflection Lists'][generalData['Name']] = {}
1532            UseList = phase['Histograms']
1533            SGData = generalData['SGData']
1534            NShkl = len(G2spc.MustrainNames(SGData))
1535            NDij = len(G2spc.HStrainNames(SGData))
1536            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1537            UseList[hist.name]['hId'] = hist.id
1538            for key, val in [('Use', True), ('LeBail', False),
1539                             ('newLeBail', True),
1540                             ('Babinet', {'BabA': [0.0, False],
1541                                          'BabU': [0.0, False]})]:
1542                if key not in UseList[hist.name]:
1543                    UseList[hist.name][key] = val
1544        else:
1545            raise RuntimeError("Unexpected histogram" + hist.name)
1546
1547
1548    def reload(self):
1549        """Reload self from self.filename"""
1550        data, names = LoadDictFromProjFile(self.filename)
1551        self.names = names
1552        # Need to deep copy the new data file data into the current tree,
1553        # so that any existing G2Phase, or G2PwdrData objects will still be
1554        # valid
1555        _deep_copy_into(from_=data, into=self.data)
1556
1557    def refine(self, newfile=None, printFile=None, makeBack=False):
1558        # TODO migrate to RefineCore
1559        # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1560        #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1561        # index_ids will automatically save the project
1562        self.index_ids()
1563        # TODO G2strMain does not properly use printFile
1564        G2strMain.Refine(self.filename, makeBack=makeBack)
1565        # Reload yourself
1566        self.reload()
1567
1568    def histogram(self, histname):
1569        """Returns the histogram named histname, or None if it does not exist.
1570
1571        :param histname: The name of the histogram (str), or ranId or index.
1572        :returns: A :class:`G2PwdrData` object, or None if
1573            the histogram does not exist
1574
1575        .. seealso::
1576            :meth:`G2Project.histograms`
1577            :meth:`G2Project.phase`
1578            :meth:`G2Project.phases`
1579            """
1580        if isinstance(histname, G2PwdrData):
1581            if histname.proj == self:
1582                return histname
1583        if histname in self.data:
1584            return G2PwdrData(self.data[histname], self)
1585        try:
1586            # see if histname is an id or ranId
1587            histname = int(histname)
1588        except ValueError:
1589            return
1590
1591        for histogram in self.histograms():
1592            if histogram.id == histname or histogram.ranId == histname:
1593                return histogram
1594
1595    def histograms(self):
1596        """Return a list of all histograms, as
1597        :class:`G2PwdrData` objects
1598
1599        .. seealso::
1600            :meth:`G2Project.histograms`
1601            :meth:`G2Project.phase`
1602            :meth:`G2Project.phases`
1603            """
1604        output = []
1605        # loop through each tree entry. If it is more than one level (more than one item in the
1606        # list of names) then it must be a histogram, unless labeled Phases or Restraints
1607        for obj in self.names:
1608            if len(obj) > 1 and obj[0] != u'Phases' and obj[0] != u'Restraints':
1609                output.append(self.histogram(obj[0]))
1610        return output
1611
1612    def phase(self, phasename):
1613        """
1614        Gives an object representing the specified phase in this project.
1615
1616        :param str phasename: A reference to the desired phase. Either the phase
1617            name (str), the phase's ranId, the phase's index (both int) or
1618            a phase object (:class:`G2Phase`)
1619        :returns: A :class:`G2Phase` object
1620        :raises: KeyError
1621
1622        .. seealso::
1623            :meth:`G2Project.histograms`
1624            :meth:`G2Project.phase`
1625            :meth:`G2Project.phases`
1626            """
1627        if isinstance(phasename, G2Phase):
1628            if phasename.proj == self:
1629                return phasename
1630        phases = self.data['Phases']
1631        if phasename in phases:
1632            return G2Phase(phases[phasename], phasename, self)
1633
1634        try:
1635            # phasename should be phase index or ranId
1636            phasename = int(phasename)
1637        except ValueError:
1638            return
1639
1640        for phase in self.phases():
1641            if phase.id == phasename or phase.ranId == phasename:
1642                return phase
1643
1644    def phases(self):
1645        """
1646        Returns a list of all the phases in the project.
1647
1648        :returns: A list of :class:`G2Phase` objects
1649
1650        .. seealso::
1651            :meth:`G2Project.histogram`
1652            :meth:`G2Project.histograms`
1653            :meth:`G2Project.phase`
1654            """
1655        for obj in self.names:
1656            if obj[0] == 'Phases':
1657                return [self.phase(p) for p in obj[1:]]
1658        return []
1659
1660    def update_ids(self):
1661        """Makes sure all phases and histograms have proper hId and pId"""
1662        # Translated from GetUsedHistogramsAndPhasesfromTree,
1663        #   GSASIIdataGUI.py:4107
1664        for i, h in enumerate(self.histograms()):
1665            h.id = i
1666        for i, p in enumerate(self.phases()):
1667            p.id = i
1668
1669    def do_refinements(self, refinements, histogram='all', phase='all',
1670                       outputnames=None, makeBack=False):
1671        """Conducts one or a series of refinements according to the
1672           input provided in parameter refinements. This is a wrapper
1673           around :meth:`iter_refinements`
1674
1675        :param list refinements: A list of dictionaries specifiying changes to be made to
1676            parameters before refinements are conducted.
1677            See the :ref:`Refinement_recipe` section for how this is defined.
1678        :param str histogram: Name of histogram for refinements to be applied
1679            to, or 'all'; note that this can be overridden for each refinement
1680            step via a "histograms" entry in the dict.
1681        :param str phase: Name of phase for refinements to be applied to, or
1682            'all'; note that this can be overridden for each refinement
1683            step via a "phases" entry in the dict.
1684        :param list outputnames: Provides a list of project (.gpx) file names
1685            to use for each refinement step (specifying None skips the save step).
1686            See :meth:`save`.
1687            Note that this can be overridden using an "output" entry in the dict.
1688        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
1689            before a refinement is performed. The default is False.
1690           
1691        To perform a single refinement without changing any parameters, use this
1692        call:
1693
1694        .. code-block::  python
1695       
1696            my_project.do_refinements([])
1697        """
1698       
1699        for proj in self.iter_refinements(refinements, histogram, phase,
1700                                          outputnames, makeBack):
1701            pass
1702        return self
1703
1704    def iter_refinements(self, refinements, histogram='all', phase='all',
1705                         outputnames=None, makeBack=False):
1706        """Conducts a series of refinements, iteratively. Stops after every
1707        refinement and yields this project, to allow error checking or
1708        logging of intermediate results. Parameter use is the same as for
1709        :meth:`do_refinements` (which calls this method).
1710
1711        >>> def checked_refinements(proj):
1712        ...     for p in proj.iter_refinements(refs):
1713        ...         # Track intermediate results
1714        ...         log(p.histogram('0').residuals)
1715        ...         log(p.phase('0').get_cell())
1716        ...         # Check if parameter diverged, nonsense answer, or whatever
1717        ...         if is_something_wrong(p):
1718        ...             raise Exception("I need a human!")
1719
1720           
1721        """
1722        if outputnames:
1723            if len(refinements) != len(outputnames):
1724                raise ValueError("Should have same number of outputs to"
1725                                 "refinements")
1726        else:
1727            outputnames = [None for r in refinements]
1728
1729        for output, refinedict in zip(outputnames, refinements):
1730            if 'histograms' in refinedict:
1731                hist = refinedict['histograms']
1732            else:
1733                hist = histogram
1734            if 'phases' in refinedict:
1735                ph = refinedict['phases']
1736            else:
1737                ph = phase
1738            if 'output' in refinedict:
1739                output = refinedict['output']
1740            self.set_refinement(refinedict, hist, ph)
1741            # Handle 'once' args - refinements that are disabled after this
1742            # refinement
1743            if 'once' in refinedict:
1744                temp = {'set': refinedict['once']}
1745                self.set_refinement(temp, hist, ph)
1746
1747            if output:
1748                self.save(output)
1749
1750            if 'skip' not in refinedict:
1751                self.refine(makeBack=makeBack)
1752            yield self
1753
1754            # Handle 'once' args - refinements that are disabled after this
1755            # refinement
1756            if 'once' in refinedict:
1757                temp = {'clear': refinedict['once']}
1758                self.set_refinement(temp, hist, ph)
1759            if 'call' in refinedict:
1760                fxn = refinedict['call']
1761                if callable(fxn):
1762                    fxn(*refinedict.get('callargs',[self]))
1763                elif callable(eval(fxn)):
1764                    eval(fxn)(*refinedict.get('callargs',[self]))
1765                else:
1766                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
1767
1768    def set_refinement(self, refinement, histogram='all', phase='all'):
1769        """Apply specified refinements to a given histogram(s) or phase(s).
1770
1771        :param dict refinement: The refinements to be conducted
1772        :param histogram: Specifies either 'all' (default), a single histogram or
1773          a list of histograms. Histograms may be specified as histogram objects
1774          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
1775          of the histogram in the project, numbered starting from 0.
1776          Omitting the parameter or the string 'all' indicates that parameters in
1777          all histograms should be set.
1778        :param phase: Specifies either 'all' (default), a single phase or
1779          a list of phases. Phases may be specified as phase objects
1780          (see :class:`G2Phase`), the phase name (str) or the index number (int)
1781          of the phase in the project, numbered starting from 0.
1782          Omitting the parameter or the string 'all' indicates that parameters in
1783          all phases should be set.
1784
1785        Note that refinement parameters are categorized as one of three types:
1786
1787        1. Histogram parameters
1788        2. Phase parameters
1789        3. Histogram-and-Phase (HAP) parameters
1790       
1791        .. seealso::
1792            :meth:`G2PwdrData.set_refinements`
1793            :meth:`G2PwdrData.clear_refinements`
1794            :meth:`G2Phase.set_refinements`
1795            :meth:`G2Phase.clear_refinements`
1796            :meth:`G2Phase.set_HAP_refinements`
1797            :meth:`G2Phase.clear_HAP_refinements`"""
1798
1799        if histogram == 'all':
1800            hists = self.histograms()
1801        elif isinstance(histogram, list) or isinstance(histogram, tuple):
1802            hists = []
1803            for h in histogram:
1804                if isinstance(h, str) or isinstance(h, int):
1805                    hists.append(self.histogram(h))
1806                else:
1807                    hists.append(h)
1808        elif isinstance(histogram, str) or isinstance(histogram, int):
1809            hists = [self.histogram(histogram)]
1810        else:
1811            hists = [histogram]
1812
1813        if phase == 'all':
1814            phases = self.phases()
1815        elif isinstance(phase, list) or isinstance(phase, tuple):
1816            phases = []
1817            for ph in phase:
1818                if isinstance(ph, str) or isinstance(ph, int):
1819                    phases.append(self.phase(ph))
1820                else:
1821                    phases.append(ph)
1822        elif isinstance(phase, str) or isinstance(phase, int):
1823            phases = [self.phase(phase)]
1824        else:
1825            phases = [phase]
1826
1827        # TODO: HAP parameters:
1828        #   Babinet
1829        #   Extinction
1830        #   HStrain
1831        #   Mustrain
1832        #   Pref. Ori
1833        #   Size
1834
1835        pwdr_set = {}
1836        phase_set = {}
1837        hap_set = {}
1838        for key, val in refinement.get('set', {}).items():
1839            # Apply refinement options
1840            if G2PwdrData.is_valid_refinement_key(key):
1841                pwdr_set[key] = val
1842            elif G2Phase.is_valid_refinement_key(key):
1843                phase_set[key] = val
1844            elif G2Phase.is_valid_HAP_refinement_key(key):
1845                hap_set[key] = val
1846            else:
1847                raise ValueError("Unknown refinement key", key)
1848
1849        for hist in hists:
1850            hist.set_refinements(pwdr_set)
1851        for phase in phases:
1852            phase.set_refinements(phase_set)
1853        for phase in phases:
1854            phase.set_HAP_refinements(hap_set, hists)
1855
1856        pwdr_clear = {}
1857        phase_clear = {}
1858        hap_clear = {}
1859        for key, val in refinement.get('clear', {}).items():
1860            # Clear refinement options
1861            if G2PwdrData.is_valid_refinement_key(key):
1862                pwdr_clear[key] = val
1863            elif G2Phase.is_valid_refinement_key(key):
1864                phase_clear[key] = val
1865            elif G2Phase.is_valid_HAP_refinement_key(key):
1866                hap_set[key] = val
1867            else:
1868                raise ValueError("Unknown refinement key", key)
1869
1870        for hist in hists:
1871            hist.clear_refinements(pwdr_clear)
1872        for phase in phases:
1873            phase.clear_refinements(phase_clear)
1874        for phase in phases:
1875            phase.clear_HAP_refinements(hap_clear, hists)
1876
1877    def index_ids(self):
1878        import GSASIIstrIO as G2strIO
1879        self.save()
1880        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
1881
1882    def add_constraint_raw(self, cons_scope, constr):
1883        """Adds a constraint of type consType to the project.
1884        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
1885
1886        WARNING it does not check the constraint is well-constructed"""
1887        constrs = self.data['Constraints']['data']
1888        if 'Global' not in constrs:
1889            constrs['Global'] = []
1890        constrs[cons_scope].append(constr)
1891
1892    def hold_many(self, vars, type):
1893        """Apply holds for all the variables in vars, for constraint of a given type.
1894
1895        type is passed directly to add_constraint_raw as consType
1896
1897        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
1898            string variable specifiers, or arguments for :meth:`make_var_obj`
1899        :param str type: A string constraint type specifier. See
1900            :class:`G2Project.add_constraint_raw`
1901
1902        """
1903        for var in vars:
1904            if isinstance(var, str):
1905                var = self.make_var_obj(var)
1906            elif not isinstance(var, G2obj.G2VarObj):
1907                var = self.make_var_obj(*var)
1908            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
1909
1910    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
1911                     reloadIdx=True):
1912        """Wrapper to create a G2VarObj. Takes either a string representaiton ("p:h:name:a")
1913        or individual names of phase, histogram, varname, and atomId.
1914
1915        Automatically converts string phase, hist, or atom names into the ID required
1916        by G2VarObj."""
1917
1918        if reloadIdx:
1919            self.index_ids()
1920
1921        # If string representation, short circuit
1922        if hist is None and varname is None and atomId is None:
1923            if isinstance(phase, str) and ':' in phase:
1924                return G2obj.G2VarObj(phase)
1925
1926        # Get phase index
1927        phaseObj = None
1928        if isinstance(phase, G2Phase):
1929            phaseObj = phase
1930            phase = G2obj.PhaseRanIdLookup[phase.ranId]
1931        elif phase in self.data['Phases']:
1932            phaseObj = self.phase(phase)
1933            phaseRanId = phaseObj.ranId
1934            phase = G2obj.PhaseRanIdLookup[phaseRanId]
1935
1936        # Get histogram index
1937        if isinstance(hist, G2PwdrData):
1938            hist = G2obj.HistRanIdLookup[hist.ranId]
1939        elif hist in self.data:
1940            histRanId = self.histogram(hist).ranId
1941            hist = G2obj.HistRanIdLookup[histRanId]
1942
1943        # Get atom index (if any)
1944        if isinstance(atomId, G2AtomRecord):
1945            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
1946        elif phaseObj:
1947            atomObj = phaseObj.atom(atomId)
1948            if atomObj:
1949                atomRanId = atomObj.ranId
1950                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
1951
1952        return G2obj.G2VarObj(phase, hist, varname, atomId)
1953
1954
1955class G2AtomRecord(G2ObjectWrapper):
1956    """Wrapper for an atom record. Has convenient accessors via @property.
1957
1958
1959    Available accessors: label, type, refinement_flags, coordinates,
1960        occupancy, ranId, id, adp_flag, uiso
1961
1962    Example:
1963
1964    >>> atom = some_phase.atom("O3")
1965    >>> # We can access the underlying data format
1966    >>> atom.data
1967    ['O3', 'O-2', '', ... ]
1968    >>> # We can also use wrapper accessors
1969    >>> atom.coordinates
1970    (0.33, 0.15, 0.5)
1971    >>> atom.refinement_flags
1972    u'FX'
1973    >>> atom.ranId
1974    4615973324315876477
1975    >>> atom.occupancy
1976    1.0
1977    """
1978    def __init__(self, data, indices, proj):
1979        self.data = data
1980        self.cx, self.ct, self.cs, self.cia = indices
1981        self.proj = proj
1982
1983    @property
1984    def label(self):
1985        return self.data[self.ct-1]
1986
1987    @property
1988    def type(self):
1989        return self.data[self.ct]
1990
1991    @property
1992    def refinement_flags(self):
1993        return self.data[self.ct+1]
1994
1995    @refinement_flags.setter
1996    def refinement_flags(self, other):
1997        # Automatically check it is a valid refinement
1998        for c in other:
1999            if c not in ' FXU':
2000                raise ValueError("Invalid atom refinement: ", other)
2001        self.data[self.ct+1] = other
2002
2003    @property
2004    def coordinates(self):
2005        return tuple(self.data[self.cx:self.cx+3])
2006
2007    @property
2008    def occupancy(self):
2009        return self.data[self.cx+3]
2010
2011    @occupancy.setter
2012    def occupancy(self, val):
2013        self.data[self.cx+3] = float(val)
2014
2015    @property
2016    def ranId(self):
2017        return self.data[self.cia+8]
2018
2019    @property
2020    def adp_flag(self):
2021        # Either 'I' or 'A'
2022        return self.data[self.cia]
2023
2024    @property
2025    def uiso(self):
2026        if self.adp_flag == 'I':
2027            return self.data[self.cia+1]
2028        else:
2029            return self.data[self.cia+2:self.cia+8]
2030
2031    @uiso.setter
2032    def uiso(self, value):
2033        if self.adp_flag == 'I':
2034            self.data[self.cia+1] = float(value)
2035        else:
2036            assert len(value) == 6
2037            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2038
2039
2040class G2PwdrData(G2ObjectWrapper):
2041    """Wraps a Powder Data Histogram."""
2042    def __init__(self, data, proj):
2043        self.data = data
2044        self.proj = proj
2045
2046    @staticmethod
2047    def is_valid_refinement_key(key):
2048        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2049                      'Instrument Parameters']
2050        return key in valid_keys
2051
2052    @property
2053    def name(self):
2054        return self.data['data'][-1]
2055
2056    @property
2057    def ranId(self):
2058        return self.data['data'][0]['ranId']
2059
2060    @property
2061    def residuals(self):
2062        '''Provides a dictionary with with the R-factors for this histogram.
2063        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2064        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2065        '''
2066        data = self.data['data'][0]
2067        return {key: data[key] for key in data
2068                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2069                   or ':' in key}
2070
2071    @property
2072    def InstrumentParameters(self):
2073        '''Provides a dictionary with with the Instrument Parameters
2074        for this histogram.
2075        '''
2076        return self.data['Instrument Parameters'][0]
2077
2078    @property
2079    def SampleParameters(self):
2080        '''Provides a dictionary with with the Sample Parameters
2081        for this histogram.
2082        '''
2083        return self.data['Sample Parameters']
2084   
2085    @property
2086    def id(self):
2087        self.proj.update_ids()
2088        return self.data['data'][0]['hId']
2089
2090    @id.setter
2091    def id(self, val):
2092        self.data['data'][0]['hId'] = val
2093
2094    def fit_fixed_points(self):
2095        """Attempts to apply a background fit to the fixed points currently specified."""
2096        def SetInstParms(Inst):
2097            dataType = Inst['Type'][0]
2098            insVary = []
2099            insNames = []
2100            insVals = []
2101            for parm in Inst:
2102                insNames.append(parm)
2103                insVals.append(Inst[parm][1])
2104                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2105                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2106                        Inst[parm][2] = False
2107            instDict = dict(zip(insNames, insVals))
2108            instDict['X'] = max(instDict['X'], 0.01)
2109            instDict['Y'] = max(instDict['Y'], 0.01)
2110            if 'SH/L' in instDict:
2111                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2112            return dataType, instDict, insVary
2113
2114        bgrnd = self.data['Background']
2115
2116        # Need our fixed points in order
2117        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2118        X = [x for x, y in bgrnd[1]['FixedPoints']]
2119        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2120
2121        limits = self.data['Limits'][1]
2122        if X[0] > limits[0]:
2123            X = [limits[0]] + X
2124            Y = [Y[0]] + Y
2125        if X[-1] < limits[1]:
2126            X += [limits[1]]
2127            Y += [Y[-1]]
2128
2129        # Some simple lookups
2130        controls = self.proj['Controls']['data']
2131        inst, inst2 = self.data['Instrument Parameters']
2132        pwddata = self.data['data'][1]
2133
2134        # Construct the data for background fitting
2135        xBeg = np.searchsorted(pwddata[0], limits[0])
2136        xFin = np.searchsorted(pwddata[0], limits[1])
2137        xdata = pwddata[0][xBeg:xFin]
2138        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2139
2140        W = [1]*len(xdata)
2141        Z = [0]*len(xdata)
2142
2143        dataType, insDict, insVary = SetInstParms(inst)
2144        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2145
2146        # Do the fit
2147        data = np.array([xdata, ydata, W, Z, Z, Z])
2148        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2149                        prevVaryList=bakVary, controls=controls)
2150
2151        # Post-fit
2152        parmDict = {}
2153        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2154        parmDict.update(bakDict)
2155        parmDict.update(insDict)
2156        pwddata[3][xBeg:xFin] *= 0
2157        pwddata[5][xBeg:xFin] *= 0
2158        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2159
2160        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2161        # TODO update background
2162        self.proj.save()
2163
2164    def getdata(self,datatype):
2165        '''Provides access to the histogram data of the selected data type
2166
2167        :param str datatype: must be one of the following values (case is ignored)
2168       
2169           * 'X': the 2theta or TOF values for the pattern
2170           * 'Yobs': the observed intensity values
2171           * 'Yweight': the weights for each data point (1/sigma**2)
2172           * 'Ycalc': the computed intensity values
2173           * 'Background': the computed background values
2174           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2175
2176        :returns: an numpy MaskedArray with data values of the requested type
2177       
2178        '''
2179        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2180        if datatype.lower() not in enums:
2181            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2182        return self.data['data'][1][enums.index(datatype.lower())]
2183       
2184    def y_calc(self):
2185        return self.data['data'][1][3]
2186
2187    def Export(self,fileroot,extension):
2188        '''Write the histogram into a file. The path is specified by fileroot and
2189        extension.
2190       
2191        :param str fileroot: name of the file, optionally with a path (extension is
2192           ignored)
2193        :param str extension: includes '.', must match an extension in global
2194           exportersByExtension['powder'] or a Exception is raised.
2195        :returns: name of file that was written
2196        '''
2197        if extension not in exportersByExtension.get('powder',[]):
2198            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2199        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2200        obj = exportersByExtension['powder'][extension]
2201        obj.SetFromArray(hist=self.data,histname=self.name)
2202        obj.Writer(self.name,fil)
2203           
2204    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2205        try:
2206            import matplotlib.pyplot as plt
2207            data = self.data['data'][1]
2208            if Yobs:
2209                plt.plot(data[0], data[1], label='Yobs')
2210            if Ycalc:
2211                plt.plot(data[0], data[3], label='Ycalc')
2212            if Background:
2213                plt.plot(data[0], data[4], label='Background')
2214            if Residual:
2215                plt.plot(data[0], data[5], label="Residual")
2216        except ImportError:
2217            pass
2218
2219    def get_wR(self):
2220        """returns the overall weighted profile R factor for a histogram
2221       
2222        :returns: a wR value as a percentage or None if not defined
2223        """
2224        return self['data'][0].get('wR')
2225
2226    def set_refinements(self, refs):
2227        """Sets the refinement parameter 'key' to the specification 'value'
2228
2229        :param dict refs: A dictionary of the parameters to be set. See
2230                          :ref:`Histogram_parameters_table` for a description of
2231                          what these dictionaries should be.
2232
2233        :returns: None
2234
2235        """
2236        do_fit_fixed_points = False
2237        for key, value in refs.items():
2238            if key == 'Limits':
2239                old_limits = self.data['Limits'][1]
2240                new_limits = value
2241                if isinstance(new_limits, dict):
2242                    if 'low' in new_limits:
2243                        old_limits[0] = new_limits['low']
2244                    if 'high' in new_limits:
2245                        old_limits[1] = new_limits['high']
2246                else:
2247                    old_limits[0], old_limits[1] = new_limits
2248            elif key == 'Sample Parameters':
2249                sample = self.data['Sample Parameters']
2250                for sparam in value:
2251                    if sparam not in sample:
2252                        raise ValueError("Unknown refinement parameter, "
2253                                         + str(sparam))
2254                    sample[sparam][1] = True
2255            elif key == 'Background':
2256                bkg, peaks = self.data['Background']
2257
2258                # If True or False, just set the refine parameter
2259                if value in (True, False):
2260                    bkg[1] = value
2261                    return
2262
2263                if 'type' in value:
2264                    bkg[0] = value['type']
2265                if 'refine' in value:
2266                    bkg[1] = value['refine']
2267                if 'no. coeffs' in value:
2268                    cur_coeffs = bkg[2]
2269                    n_coeffs = value['no. coeffs']
2270                    if n_coeffs > cur_coeffs:
2271                        for x in range(n_coeffs - cur_coeffs):
2272                            bkg.append(0.0)
2273                    else:
2274                        for _ in range(cur_coeffs - n_coeffs):
2275                            bkg.pop()
2276                    bkg[2] = n_coeffs
2277                if 'coeffs' in value:
2278                    bkg[3:] = value['coeffs']
2279                if 'FixedPoints' in value:
2280                    peaks['FixedPoints'] = [(float(a), float(b))
2281                                            for a, b in value['FixedPoints']]
2282                if value.get('fit fixed points', False):
2283                    do_fit_fixed_points = True
2284
2285            elif key == 'Instrument Parameters':
2286                instrument, secondary = self.data['Instrument Parameters']
2287                for iparam in value:
2288                    try:
2289                        instrument[iparam][2] = True
2290                    except IndexError:
2291                        raise ValueError("Invalid key:", iparam)
2292            else:
2293                raise ValueError("Unknown key:", key)
2294        # Fit fixed points after the fact - ensure they are after fixed points
2295        # are added
2296        if do_fit_fixed_points:
2297            # Background won't be fit if refinement flag not set
2298            orig = self.data['Background'][0][1]
2299            self.data['Background'][0][1] = True
2300            self.fit_fixed_points()
2301            # Restore the previous value
2302            self.data['Background'][0][1] = orig
2303
2304    def clear_refinements(self, refs):
2305        """Clears the refinement parameter 'key' and its associated value.
2306
2307        :param dict refs: A dictionary of parameters to clear."""
2308        for key, value in refs.items():
2309            if key == 'Limits':
2310                old_limits, cur_limits = self.data['Limits']
2311                cur_limits[0], cur_limits[1] = old_limits
2312            elif key == 'Sample Parameters':
2313                sample = self.data['Sample Parameters']
2314                for sparam in value:
2315                    sample[sparam][1] = False
2316            elif key == 'Background':
2317                bkg, peaks = self.data['Background']
2318
2319                # If True or False, just set the refine parameter
2320                if value in (True, False):
2321                    bkg[1] = False
2322                    return
2323
2324                bkg[1] = False
2325                if 'FixedPoints' in value:
2326                    if 'FixedPoints' in peaks:
2327                        del peaks['FixedPoints']
2328            elif key == 'Instrument Parameters':
2329                instrument, secondary = self.data['Instrument Parameters']
2330                for iparam in value:
2331                    instrument[iparam][2] = False
2332            else:
2333                raise ValueError("Unknown key:", key)
2334
2335    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2336        '''Adds a single peak to the peak list
2337        :param float area: peak area
2338        :param float dspace: peak position as d-space (A)
2339        :param float Q: peak position as Q (A-1)
2340        :param float ttheta: peak position as 2Theta (deg)
2341
2342        Note: only one of the parameters dspace, Q or ttheta may be specified
2343        '''
2344        import GSASIIlattice as G2lat
2345        import GSASIImath as G2mth
2346        if (not dspace) + (not Q) + (not ttheta) != 2:
2347            print('add_peak error: too many or no peak position(s) specified')
2348            return
2349        pos = ttheta
2350        Parms,Parms2 = self.data['Instrument Parameters']
2351        if Q:
2352            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2353        elif dspace:
2354            pos = G2lat.Dsp2pos(Parms,dspace)
2355        peaks = self.data['Peak List']
2356        peaks['sigDict'] = {}        #no longer valid
2357        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2358
2359    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2360        '''Set refinement flags for peaks
2361       
2362        :param list peaklist: a list of peaks to change flags. If None (default), changes
2363          are made to all peaks.
2364        :param bool area: Sets or clears the refinement flag for the peak area value.
2365          If None (the default), no change is made.
2366        :param bool pos: Sets or clears the refinement flag for the peak position value.
2367          If None (the default), no change is made.
2368        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2369          If None (the default), no change is made.
2370        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2371          If None (the default), no change is made.
2372         
2373        Note that when peaks are first created the area flag is on and the other flags are
2374        initially off.
2375
2376        Example::
2377       
2378           set_peakFlags(sig=False,gam=True)
2379
2380        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
2381        all peaks. The position and area flags are not changed from their previous values.
2382        '''
2383        peaks = self.data['Peak List']
2384        if peaklist is None:
2385            peaklist = range(len(peaks['peaks']))
2386        for i in peaklist:
2387            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
2388                if var is not None:
2389                    peaks['peaks'][i][j] = var
2390           
2391    def refine_peaks(self):
2392        '''Causes a refinement of peak position, background and instrument parameters
2393        '''
2394        import GSASIIpwd as G2pwd
2395        controls = self.proj.data.get('Controls',{})
2396        controls = controls.get('data',
2397                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
2398                            )
2399        peaks = self.data['Peak List']
2400        Parms,Parms2 = self.data['Instrument Parameters']
2401        background = self.data['Background']
2402        limits = self.data['Limits'][1]
2403        bxye = np.zeros(len(self.data['data'][1][1]))
2404        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
2405                                           Parms,Parms2,self.data['data'][1],bxye,[],
2406                                           False,controls,None)[0]
2407
2408    def SaveProfile(self,filename):
2409        '''Writes a GSAS-II (new style) .instprm file
2410        '''
2411        data,Parms2 = self.data['Instrument Parameters']
2412        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2413        File = open(filename,'w')
2414        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
2415        for item in data:
2416            File.write(item+':'+str(data[item][1])+'\n')
2417        File.close()
2418        print ('Instrument parameters saved to: '+filename)
2419
2420    def LoadProfile(self,filename):
2421        '''Reads a GSAS-II (new style) .instprm file and overwrites the current parameters
2422        '''
2423        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
2424        File = open(filename,'r')
2425        S = File.readline()
2426        newItems = []
2427        newVals = []
2428        Found = False
2429        while S:
2430            if S[0] == '#':
2431                if Found:
2432                    break
2433                if 'Bank' in S:
2434                    if bank == int(S.split(':')[0].split()[1]):
2435                        S = File.readline()
2436                        continue
2437                    else:
2438                        S = File.readline()
2439                        while S and '#Bank' not in S:
2440                            S = File.readline()
2441                        continue
2442                else:   #a non #Bank file
2443                    S = File.readline()
2444                    continue
2445            Found = True
2446            [item,val] = S[:-1].split(':')
2447            newItems.append(item)
2448            try:
2449                newVals.append(float(val))
2450            except ValueError:
2451                newVals.append(val)                       
2452            S = File.readline()               
2453        File.close()
2454        LoadG2fil()
2455        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
2456
2457       
2458class G2Phase(G2ObjectWrapper):
2459    """A wrapper object around a given phase.
2460
2461    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
2462    """
2463    def __init__(self, data, name, proj):
2464        self.data = data
2465        self.name = name
2466        self.proj = proj
2467
2468    @staticmethod
2469    def is_valid_refinement_key(key):
2470        valid_keys = ["Cell", "Atoms", "LeBail"]
2471        return key in valid_keys
2472
2473    @staticmethod
2474    def is_valid_HAP_refinement_key(key):
2475        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
2476                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
2477        return key in valid_keys
2478
2479    def atom(self, atomlabel):
2480        """Returns the atom specified by atomlabel, or None if it does not
2481        exist.
2482
2483        :param str atomlabel: The name of the atom (e.g. "O2")
2484        :returns: A :class:`G2AtomRecord` object
2485            representing the atom.
2486        """
2487        # Consult GSASIIobj.py for the meaning of this
2488        cx, ct, cs, cia = self.data['General']['AtomPtrs']
2489        ptrs = [cx, ct, cs, cia]
2490        atoms = self.data['Atoms']
2491        for atom in atoms:
2492            if atom[ct-1] == atomlabel:
2493                return G2AtomRecord(atom, ptrs, self.proj)
2494
2495    def atoms(self):
2496        """Returns a list of atoms present in the phase.
2497
2498        :returns: A list of :class:`G2AtomRecord` objects.
2499
2500        .. seealso::
2501            :meth:`G2Phase.atom`
2502            :class:`G2AtomRecord`
2503        """
2504        ptrs = self.data['General']['AtomPtrs']
2505        output = []
2506        atoms = self.data['Atoms']
2507        for atom in atoms:
2508            output.append(G2AtomRecord(atom, ptrs, self.proj))
2509        return output
2510
2511    def histograms(self):
2512        output = []
2513        for hname in self.data.get('Histograms', {}).keys():
2514            output.append(self.proj.histogram(hname))
2515        return output
2516
2517    @property
2518    def ranId(self):
2519        return self.data['ranId']
2520
2521    @property
2522    def id(self):
2523        return self.data['pId']
2524
2525    @id.setter
2526    def id(self, val):
2527        self.data['pId'] = val
2528
2529    def get_cell(self):
2530        """Returns a dictionary of the cell parameters, with keys:
2531            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
2532
2533        :returns: a dict
2534
2535        .. seealso::
2536           :meth:`G2Phase.get_cell_and_esd`
2537
2538        """
2539        cell = self.data['General']['Cell']
2540        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
2541                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
2542                'volume': cell[7]}
2543
2544    def get_cell_and_esd(self):
2545        """
2546        Returns a pair of dictionaries, the first representing the unit cell, the second
2547        representing the estimated standard deviations of the unit cell.
2548
2549        :returns: a tuple of two dictionaries
2550
2551        .. seealso::
2552           :meth:`G2Phase.get_cell`
2553
2554        """
2555        # translated from GSASIIstrIO.ExportBaseclass.GetCell
2556        import GSASIIstrIO as G2stIO
2557        import GSASIIlattice as G2lat
2558        import GSASIImapvars as G2mv
2559        try:
2560            pfx = str(self.id) + '::'
2561            sgdata = self['General']['SGData']
2562            covDict = self.proj['Covariance']['data']
2563
2564            parmDict = dict(zip(covDict.get('varyList',[]),
2565                                covDict.get('variables',[])))
2566            sigDict = dict(zip(covDict.get('varyList',[]),
2567                               covDict.get('sig',[])))
2568
2569            if covDict.get('covMatrix') is not None:
2570                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2571                                                  covDict['varyList'],
2572                                                  parmDict))
2573
2574            A, sigA = G2stIO.cellFill(pfx, sgdata, parmDict, sigDict)
2575            cellSig = G2stIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
2576            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
2577            cellDict, cellSigDict = {}, {}
2578            for i, key in enumerate(['length_a', 'length_b', 'length_c',
2579                                     'angle_alpha', 'angle_beta', 'angle_gamma',
2580                                     'volume']):
2581                cellDict[key] = cellList[i]
2582                cellSigDict[key] = cellSig[i]
2583            return cellDict, cellSigDict
2584        except KeyError:
2585            cell = self.get_cell()
2586            return cell, {key: 0.0 for key in cell}
2587
2588    def export_CIF(self, outputname, quickmode=True):
2589        """Write this phase to a .cif file named outputname
2590
2591        :param str outputname: The name of the .cif file to write to
2592        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
2593        # This code is all taken from exports/G2export_CIF.py
2594        # Functions copied have the same names
2595        import GSASIImath as G2mth
2596        import GSASIImapvars as G2mv
2597        from exports import G2export_CIF as cif
2598
2599        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
2600        CIFname = os.path.splitext(self.proj.filename)[0]
2601        CIFname = os.path.split(CIFname)[1]
2602        CIFname = ''.join([c if ord(c) < 128 else ''
2603                           for c in CIFname.replace(' ', '_')])
2604        try:
2605            author = self.proj['Controls']['data'].get('Author','').strip()
2606        except KeyError:
2607            pass
2608        oneblock = True
2609
2610        covDict = self.proj['Covariance']['data']
2611        parmDict = dict(zip(covDict.get('varyList',[]),
2612                            covDict.get('variables',[])))
2613        sigDict = dict(zip(covDict.get('varyList',[]),
2614                           covDict.get('sig',[])))
2615
2616        if covDict.get('covMatrix') is not None:
2617            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
2618                                              covDict['varyList'],
2619                                              parmDict))
2620
2621        with open(outputname, 'w') as fp:
2622            fp.write(' \n' + 70*'#' + '\n')
2623            cif.WriteCIFitem(fp, 'data_' + CIFname)
2624            # from exports.G2export_CIF.WritePhaseInfo
2625            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
2626            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
2627            # TODO get esds
2628            cellDict = self.get_cell()
2629            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
2630            names = ['length_a','length_b','length_c',
2631                     'angle_alpha','angle_beta ','angle_gamma',
2632                     'volume']
2633            for key, val in cellDict.items():
2634                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
2635
2636            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
2637                         self.data['General']['SGData']['SGSys'])
2638
2639            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
2640            # regularize capitalization and remove trailing H/R
2641            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
2642            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
2643
2644            # generate symmetry operations including centering and center of symmetry
2645            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
2646                self.data['General']['SGData'])
2647            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
2648            for i, op in enumerate(SymOpList,start=1):
2649                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
2650
2651            # TODO skipped histograms, exports/G2export_CIF.py:880
2652
2653            # report atom params
2654            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
2655                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
2656                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
2657            else:
2658                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
2659            # report cell contents
2660            cif.WriteComposition(fp, self.data, self.name, parmDict)
2661            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
2662                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
2663                raise NotImplementedError("only quickmode currently supported")
2664            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
2665                cif.WriteCIFitem(fp,'\n# Difference density results')
2666                MinMax = self.data['General']['Map']['minmax']
2667                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
2668                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
2669
2670
2671    def set_refinements(self, refs):
2672        """Sets the refinement parameter 'key' to the specification 'value'
2673
2674        :param dict refs: A dictionary of the parameters to be set. See
2675                          :ref:`Phase_parameters_table` for a description of
2676                          this dictionary.
2677
2678        :returns: None"""
2679        for key, value in refs.items():
2680            if key == "Cell":
2681                self.data['General']['Cell'][0] = value
2682
2683            elif key == "Atoms":
2684                for atomlabel, atomrefinement in value.items():
2685                    if atomlabel == 'all':
2686                        for atom in self.atoms():
2687                            atom.refinement_flags = atomrefinement
2688                    else:
2689                        atom = self.atom(atomlabel)
2690                        if atom is None:
2691                            raise ValueError("No such atom: " + atomlabel)
2692                        atom.refinement_flags = atomrefinement
2693
2694            elif key == "LeBail":
2695                hists = self.data['Histograms']
2696                for hname, hoptions in hists.items():
2697                    if 'LeBail' not in hoptions:
2698                        hoptions['newLeBail'] = bool(True)
2699                    hoptions['LeBail'] = bool(value)
2700            else:
2701                raise ValueError("Unknown key:", key)
2702
2703    def clear_refinements(self, refs):
2704        """Clears a given set of parameters.
2705
2706        :param dict refs: The parameters to clear"""
2707        for key, value in refs.items():
2708            if key == "Cell":
2709                self.data['General']['Cell'][0] = False
2710            elif key == "Atoms":
2711                cx, ct, cs, cia = self.data['General']['AtomPtrs']
2712
2713                for atomlabel in value:
2714                    atom = self.atom(atomlabel)
2715                    # Set refinement to none
2716                    atom.refinement_flags = ' '
2717            elif key == "LeBail":
2718                hists = self.data['Histograms']
2719                for hname, hoptions in hists.items():
2720                    if 'LeBail' not in hoptions:
2721                        hoptions['newLeBail'] = True
2722                    hoptions['LeBail'] = False
2723            else:
2724                raise ValueError("Unknown key:", key)
2725
2726    def set_HAP_refinements(self, refs, histograms='all'):
2727        """Sets the given HAP refinement parameters between this phase and
2728        the given histograms
2729
2730        :param dict refs: A dictionary of the parameters to be set. See
2731                          :ref:`HAP_parameters_table` for a description of this
2732                          dictionary.
2733        :param histograms: Either 'all' (default) or a list of the histograms
2734            whose HAP parameters will be set with this phase. Histogram and phase
2735            must already be associated
2736
2737        :returns: None
2738        """
2739        if histograms == 'all':
2740            histograms = self.data['Histograms'].values()
2741        else:
2742            histograms = [self.data['Histograms'][h.name] for h in histograms]
2743
2744        for key, val in refs.items():
2745            for h in histograms:
2746                if key == 'Babinet':
2747                    try:
2748                        sets = list(val)
2749                    except ValueError:
2750                        sets = ['BabA', 'BabU']
2751                    for param in sets:
2752                        if param not in ['BabA', 'BabU']:
2753                            raise ValueError("Not sure what to do with" + param)
2754                        for hist in histograms:
2755                            hist['Babinet'][param][1] = True
2756                elif key == 'Extinction':
2757                    for h in histograms:
2758                        h['Extinction'][1] = bool(val)
2759                elif key == 'HStrain':
2760                    for h in histograms:
2761                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
2762                elif key == 'Mustrain':
2763                    for h in histograms:
2764                        mustrain = h['Mustrain']
2765                        newType = None
2766                        direction = None
2767                        if isinstance(val, strtypes):
2768                            if val in ['isotropic', 'uniaxial', 'generalized']:
2769                                newType = val
2770                            else:
2771                                raise ValueError("Not a Mustrain type: " + val)
2772                        elif isinstance(val, dict):
2773                            newType = val.get('type', None)
2774                            direction = val.get('direction', None)
2775
2776                        if newType:
2777                            mustrain[0] = newType
2778                            if newType == 'isotropic':
2779                                mustrain[2][0] = True == val.get('refine',False)
2780                                mustrain[5] = [False for p in mustrain[4]]
2781                            elif newType == 'uniaxial':
2782                                if 'refine' in val:
2783                                    mustrain[2][0] = False
2784                                    types = val['refine']
2785                                    if isinstance(types, strtypes):
2786                                        types = [types]
2787                                    elif isinstance(types, bool):
2788                                        mustrain[2][1] = types
2789                                        mustrain[2][2] = types
2790                                        types = []
2791                                    else:
2792                                        raise ValueError("Not sure what to do with: "
2793                                                         + str(types))
2794                                else:
2795                                    types = []
2796
2797                                for unitype in types:
2798                                    if unitype == 'equatorial':
2799                                        mustrain[2][0] = True
2800                                    elif unitype == 'axial':
2801                                        mustrain[2][1] = True
2802                                    else:
2803                                        msg = 'Invalid uniaxial mustrain type'
2804                                        raise ValueError(msg + ': ' + unitype)
2805                            else:  # newtype == 'generalized'
2806                                mustrain[2] = [False for p in mustrain[1]]
2807                                if 'refine' in val:
2808                                    mustrain[5] = [True == val['refine']]*len(mustrain[5])
2809
2810                        if direction:
2811                            if len(direction) != 3:
2812                                raise ValueError("Expected hkl, found", direction)
2813                            direction = [int(n) for n in direction]
2814                            mustrain[3] = direction
2815                elif key == 'Size':
2816                    newSize = None
2817                    if 'value' in val:
2818                        newSize = float(val['value'])
2819                    for h in histograms:
2820                        size = h['Size']
2821                        newType = None
2822                        direction = None
2823                        if isinstance(val, strtypes):
2824                            if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
2825                                newType = val
2826                            else:
2827                                raise ValueError("Not a valid Size type: " + val)
2828                        elif isinstance(val, dict):
2829                            newType = val.get('type', None)
2830                            direction = val.get('direction', None)
2831
2832                        if newType:
2833                            size[0] = newType
2834                            refine = True == val.get('refine')
2835                            if newType == 'isotropic' and refine is not None:
2836                                size[2][0] = bool(refine)
2837                                if newSize: size[1][0] = newSize
2838                            elif newType == 'uniaxial' and refine is not None:
2839                                size[2][1] = bool(refine)
2840                                size[2][2] = bool(refine)
2841                                if newSize: size[1][1] = size[1][2] =newSize
2842                            elif newType == 'ellipsoidal' and refine is not None:
2843                                size[5] = [bool(refine) for p in size[5]]
2844                                if newSize: size[4] = [newSize for p in size[4]]
2845
2846                        if direction:
2847                            if len(direction) != 3:
2848                                raise ValueError("Expected hkl, found", direction)
2849                            direction = [int(n) for n in direction]
2850                            size[3] = direction
2851                elif key == 'Pref.Ori.':
2852                    for h in histograms:
2853                        h['Pref.Ori.'][2] = bool(val)
2854                elif key == 'Show':
2855                    for h in histograms:
2856                        h['Show'] = bool(val)
2857                elif key == 'Use':
2858                    for h in histograms:
2859                        h['Use'] = bool(val)
2860                elif key == 'Scale':
2861                    for h in histograms:
2862                        h['Scale'][1] = bool(val)
2863                else:
2864                    print(u'Unknown HAP key: '+key)
2865
2866    def getHAPvalues(self, histname):
2867        """Returns a dict with HAP values for the selected histogram
2868
2869        :param histogram: is a histogram object (:class:`G2PwdrData`) or
2870            a histogram name or the index number of the histogram
2871
2872        :returns: HAP value dict
2873        """
2874        if isinstance(histname, G2PwdrData):
2875            histname = histname.name
2876        elif histname in self.data['Histograms']:
2877            pass
2878        elif type(histname) is int:
2879            histname = self.proj.histograms()[histname].name
2880        else:
2881            raise G2ScriptException("Invalid histogram reference: "+str(histname))
2882        return self.data['Histograms'][histname]
2883                   
2884    def clear_HAP_refinements(self, refs, histograms='all'):
2885        """Clears the given HAP refinement parameters between this phase and
2886        the given histograms
2887
2888        :param dict refs: A dictionary of the parameters to be cleared.
2889        :param histograms: Either 'all' (default) or a list of the histograms
2890            whose HAP parameters will be cleared with this phase. Histogram and
2891            phase must already be associated
2892
2893        :returns: None
2894        """
2895        if histograms == 'all':
2896            histograms = self.data['Histograms'].values()
2897        else:
2898            histograms = [self.data['Histograms'][h.name] for h in histograms]
2899
2900        for key, val in refs.items():
2901            for h in histograms:
2902                if key == 'Babinet':
2903                    try:
2904                        sets = list(val)
2905                    except ValueError:
2906                        sets = ['BabA', 'BabU']
2907                    for param in sets:
2908                        if param not in ['BabA', 'BabU']:
2909                            raise ValueError("Not sure what to do with" + param)
2910                        for hist in histograms:
2911                            hist['Babinet'][param][1] = False
2912                elif key == 'Extinction':
2913                    for h in histograms:
2914                        h['Extinction'][1] = False
2915                elif key == 'HStrain':
2916                    for h in histograms:
2917                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
2918                elif key == 'Mustrain':
2919                    for h in histograms:
2920                        mustrain = h['Mustrain']
2921                        mustrain[2] = [False for p in mustrain[2]]
2922                        mustrain[5] = [False for p in mustrain[4]]
2923                elif key == 'Pref.Ori.':
2924                    for h in histograms:
2925                        h['Pref.Ori.'][2] = False
2926                elif key == 'Show':
2927                    for h in histograms:
2928                        h['Show'] = False
2929                elif key == 'Size':
2930                    for h in histograms:
2931                        size = h['Size']
2932                        size[2] = [False for p in size[2]]
2933                        size[5] = [False for p in size[5]]
2934                elif key == 'Use':
2935                    for h in histograms:
2936                        h['Use'] = False
2937                elif key == 'Scale':
2938                    for h in histograms:
2939                        h['Scale'][1] = False
2940                else:
2941                    print(u'Unknown HAP key: '+key)
2942
2943
2944##########################
2945# Command Line Interface #
2946##########################
2947# Each of these takes an argparse.Namespace object as their argument,
2948# representing the parsed command-line arguments for the relevant subcommand.
2949# The argument specification for each is in the subcommands dictionary (see
2950# below)
2951
2952commandhelp={}
2953commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
2954def create(args):
2955    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
2956
2957  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
2958                                  [-i IPARAMS [IPARAMS ...]]
2959                                  [-p PHASES [PHASES ...]]
2960                                  filename
2961                                 
2962positional arguments::
2963
2964  filename              the project file to create. should end in .gpx
2965
2966optional arguments::
2967
2968  -h, --help            show this help message and exit
2969  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
2970                        list of datafiles to add as histograms
2971  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
2972                        instrument parameter file, must be one for every
2973                        histogram
2974  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
2975                        list of phases to add. phases are automatically
2976                        associated with all histograms given.
2977
2978    """
2979    proj = G2Project(gpxname=args.filename)
2980
2981    hist_objs = []
2982    if args.histograms:
2983        for h,i in zip(args.histograms,args.iparams):
2984            print("Adding histogram from",h,"with instparm ",i)
2985            hist_objs.append(proj.add_powder_histogram(h, i))
2986
2987    if args.phases: 
2988        for p in args.phases:
2989            print("Adding phase from",p)
2990            proj.add_phase(p, histograms=hist_objs)
2991        print('Linking phase(s) to histogram(s):')
2992        for h in hist_objs:
2993            print ('   '+h.name)
2994
2995    proj.save()
2996
2997commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
2998def add(args):
2999    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
3000
3001  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
3002                               [-i IPARAMS [IPARAMS ...]]
3003                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
3004                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
3005                               filename
3006
3007
3008positional arguments::
3009
3010  filename              the project file to open. Should end in .gpx
3011
3012optional arguments::
3013
3014  -h, --help            show this help message and exit
3015  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
3016                        list of datafiles to add as histograms
3017  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
3018                        instrument parameter file, must be one for every
3019                        histogram
3020  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
3021                        format hint for histogram import. Applies to all
3022                        histograms
3023  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
3024                        list of phases to add. phases are automatically
3025                        associated with all histograms given.
3026  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
3027                        format hint for phase import. Applies to all phases.
3028                        Example: -pf CIF
3029  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
3030                        list of histgram indices to associate with added
3031                        phases. If not specified, phases are associated with
3032                        all previously loaded histograms. Example: -l 2 3 4
3033   
3034    """
3035    proj = G2Project(args.filename)
3036
3037    if args.histograms:
3038        for h,i in zip(args.histograms,args.iparams):
3039            print("Adding histogram from",h,"with instparm ",i)
3040            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
3041
3042    if args.phases: 
3043        if not args.histlist:
3044            histlist = proj.histograms()
3045        else:
3046            histlist = [proj.histogram(i) for i in args.histlist]
3047
3048        for p in args.phases:
3049            print("Adding phase from",p)
3050            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
3051           
3052        if not args.histlist:
3053            print('Linking phase(s) to all histogram(s)')
3054        else:
3055            print('Linking phase(s) to histogram(s):')
3056            for h in histlist:
3057                print ('   '+h.name)
3058
3059    proj.save()
3060
3061
3062commandhelp["dump"] = "Shows the contents of a GSAS-II project"
3063def dump(args):
3064    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
3065
3066       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
3067
3068positional arguments::
3069
3070  files
3071
3072optional arguments::
3073
3074  -h, --help        show this help message and exit
3075  -d, --histograms  list histograms in files, overrides --raw
3076  -p, --phases      list phases in files, overrides --raw
3077  -r, --raw         dump raw file contents, default
3078 
3079    """
3080    if not args.histograms and not args.phases:
3081        args.raw = True
3082    if args.raw:
3083        import IPython.lib.pretty as pretty
3084
3085    for fname in args.files:
3086        if args.raw:
3087            proj, nameList = LoadDictFromProjFile(fname)
3088            print("file:", fname)
3089            print("names:", nameList)
3090            for key, val in proj.items():
3091                print(key, ":")
3092                pretty.pprint(val)
3093        else:
3094            proj = G2Project(fname)
3095            if args.histograms:
3096                hists = proj.histograms()
3097                for h in hists:
3098                    print(fname, "hist", h.id, h.name)
3099            if args.phases:
3100                phase_list = proj.phases()
3101                for p in phase_list:
3102                    print(fname, "phase", p.id, p.name)
3103
3104
3105commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
3106def IPyBrowse(args):
3107    """Load a .gpx file and then open a IPython shell to browse it::
3108
3109  usage: GSASIIscriptable.py browse [-h] files [files ...]
3110
3111positional arguments::
3112
3113  files       list of files to browse
3114
3115optional arguments::
3116
3117  -h, --help  show this help message and exit
3118
3119    """
3120    for fname in args.files:
3121        proj, nameList = LoadDictFromProjFile(fname)
3122        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
3123        GSASIIpath.IPyBreak_base(msg)
3124        break
3125
3126
3127commandhelp["refine"] = '''
3128Conducts refinements on GSAS-II projects according to a list of refinement
3129steps in a JSON dict
3130'''
3131def refine(args):
3132    """Implements the refine command-line subcommand:
3133    conducts refinements on GSAS-II projects according to a JSON refinement dict::
3134
3135        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
3136
3137positional arguments::
3138
3139  gpxfile      the project file to refine
3140  refinements  json file of refinements to apply. if not present refines file
3141               as-is
3142
3143optional arguments::
3144
3145  -h, --help   show this help message and exit
3146 
3147    """
3148    proj = G2Project(args.gpxfile)
3149    if args.refinements is None:
3150        proj.refine()
3151    else:
3152        import json
3153        with open(args.refinements) as refs:
3154            refs = json.load(refs)
3155        if type(refs) is not dict:
3156            raise G2ScriptException("Error: JSON object must be a dict.")
3157        if "code" in refs:
3158            print("executing code:\n|  ",'\n|  '.join(refs['code']))
3159            exec('\n'.join(refs['code']))
3160        proj.do_refinements(refs['refinements'])
3161
3162
3163commandhelp["seqrefine"] = "Not implemented. Placeholder for eventual sequential refinement implementation"
3164def seqrefine(args):
3165    """Future implementation for the seqrefine command-line subcommand """
3166    raise NotImplementedError("seqrefine is not yet implemented")
3167
3168
3169commandhelp["export"] = "Export phase as CIF"
3170def export(args):
3171    """Implements the export command-line subcommand: Exports phase as CIF::
3172
3173      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
3174
3175positional arguments::
3176
3177  gpxfile     the project file from which to export
3178  phase       identifier of phase to export
3179  exportfile  the .cif file to export to
3180
3181optional arguments::
3182
3183  -h, --help  show this help message and exit
3184
3185    """
3186    proj = G2Project(args.gpxfile)
3187    phase = proj.phase(args.phase)
3188    phase.export_CIF(args.exportfile)
3189
3190
3191def _args_kwargs(*args, **kwargs):
3192    return args, kwargs
3193
3194# A dictionary of the name of each subcommand, and a tuple
3195# of its associated function and a list of its arguments
3196# The arguments are passed directly to the add_argument() method
3197# of an argparse.ArgumentParser
3198
3199subcommands = {"create":
3200               (create, [_args_kwargs('filename',
3201                                      help='the project file to create. should end in .gpx'),
3202
3203                         _args_kwargs('-d', '--histograms',
3204                                      nargs='+',
3205                                      help='list of datafiles to add as histograms'),
3206                                     
3207                         _args_kwargs('-i', '--iparams',
3208                                      nargs='+',
3209                                      help='instrument parameter file, must be one'
3210                                           ' for every histogram'
3211                                      ),
3212
3213                         _args_kwargs('-p', '--phases',
3214                                      nargs='+',
3215                                      help='list of phases to add. phases are '
3216                                           'automatically associated with all '
3217                                           'histograms given.')]),
3218               "add": (add, [_args_kwargs('filename',
3219                                      help='the project file to open. Should end in .gpx'),
3220
3221                         _args_kwargs('-d', '--histograms',
3222                                      nargs='+',
3223                                      help='list of datafiles to add as histograms'),
3224                                     
3225                         _args_kwargs('-i', '--iparams',
3226                                      nargs='+',
3227                                      help='instrument parameter file, must be one'
3228                                           ' for every histogram'
3229                                      ),
3230                                     
3231                         _args_kwargs('-hf', '--histogramformat',
3232                                      help='format hint for histogram import. Applies to all'
3233                                           ' histograms'
3234                                      ),
3235
3236                         _args_kwargs('-p', '--phases',
3237                                      nargs='+',
3238                                      help='list of phases to add. phases are '
3239                                           'automatically associated with all '
3240                                           'histograms given.'),
3241
3242                         _args_kwargs('-pf', '--phaseformat',
3243                                      help='format hint for phase import. Applies to all'
3244                                           ' phases. Example: -pf CIF'
3245                                      ),
3246                                     
3247                         _args_kwargs('-l', '--histlist',
3248                                      nargs='+',
3249                                      help='list of histgram indices to associate with added'
3250                                           ' phases. If not specified, phases are'
3251                                           ' associated with all previously loaded'
3252                                           ' histograms. Example: -l 2 3 4')]),
3253                                           
3254               "dump": (dump, [_args_kwargs('-d', '--histograms',
3255                                     action='store_true',
3256                                     help='list histograms in files, overrides --raw'),
3257
3258                               _args_kwargs('-p', '--phases',
3259                                            action='store_true',
3260                                            help='list phases in files, overrides --raw'),
3261
3262                               _args_kwargs('-r', '--raw',
3263                                      action='store_true', help='dump raw file contents, default'),
3264
3265                               _args_kwargs('files', nargs='+')]),
3266
3267               "refine":
3268               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
3269                         _args_kwargs('refinements',
3270                                      help='JSON file of refinements to apply. if not present'
3271                                           ' refines file as-is',
3272                                      default=None,
3273                                      nargs='?')]),
3274
3275               "seqrefine": (seqrefine, []),
3276               "export": (export, [_args_kwargs('gpxfile',
3277                                                help='the project file from which to export'),
3278                                   _args_kwargs('phase', help='identifier of phase to export'),
3279                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
3280               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
3281                                                   help='list of files to browse')])}
3282
3283
3284def main():
3285    '''The command-line interface for calling GSASIIscriptable as a shell command,
3286    where it is expected to be called as::
3287
3288       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
3289
3290    The following subcommands are defined:
3291
3292        * create, see :func:`create`
3293        * add, see :func:`add`
3294        * dump, see :func:`dump`
3295        * refine, see :func:`refine`
3296        * seqrefine, see :func:`seqrefine`
3297        * export, :func:`export`
3298        * browse, see :func:`IPyBrowse`
3299
3300    .. seealso::
3301        :func:`create`
3302        :func:`add`
3303        :func:`dump`
3304        :func:`refine`
3305        :func:`seqrefine`
3306        :func:`export`
3307        :func:`IPyBrowse`
3308    '''
3309    parser = argparse.ArgumentParser(description=
3310        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
3311        )
3312    subs = parser.add_subparsers()
3313
3314    # Create all of the specified subparsers
3315    for name, (func, args) in subcommands.items():
3316        new_parser = subs.add_parser(name,help=commandhelp.get(name),
3317                                     description='Command "'+name+'" '+commandhelp.get(name))
3318        for listargs, kwds in args:
3319            new_parser.add_argument(*listargs, **kwds)
3320        new_parser.set_defaults(func=func)
3321
3322    # Parse and trigger subcommand
3323    result = parser.parse_args()
3324    result.func(result)
3325
3326if __name__ == '__main__':
3327    #fname='/tmp/corundum-template.gpx'
3328    #prj = G2Project(fname)
3329    main()
Note: See TracBrowser for help on using the repository browser.