source: trunk/GSASIIscriptable.py @ 3970

Last change on this file since 3970 was 3970, checked in by toby, 2 years ago

Py3 bug on Pawley Estimate; Scriptable corrections

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 190.8 KB
Line 
1#!/usr/bin/env python
2# -*- coding: utf-8 -*-
3########### SVN repository information ###################
4# $Date: 2019-05-12 15:57:38 +0000 (Sun, 12 May 2019) $
5# $Author: toby $
6# $Revision: 3970 $
7# $URL: trunk/GSASIIscriptable.py $
8# $Id: GSASIIscriptable.py 3970 2019-05-12 15:57:38Z toby $
9########### SVN repository information ###################
10#
11"""
12*GSASIIscriptable: Scripting Interface*
13=======================================
14
15Routines to use an increasing amount of GSAS-II's capabilities from scripts,
16without use of the graphical user interface (GUI). GSASIIscriptable can create and access
17GSAS-II project (.gpx) files and can directly perform image handling and refinements. 
18The module defines wrapper classes (inheriting from :class:`G2ObjectWrapper`) for a growing number
19of data tree items.
20
21GSASIIscriptable can be used in two ways. It offers a command-line mode
22(see :ref:`CommandlineInterface`) that
23provides access a number of features without writing Python scripts
24via shell/batch commands. The more powerful mode of GSASIIscriptable is
25use is through Python scripts that
26call the module's application interface (API), see API summary that follows or the :ref:`API`
27section.
28
29==================================================
30Application Interface (API) Summary
31==================================================
32This section of the documentation provides an overview to API, with full documentation
33in the :ref:`API` section. The typical API use will be with a Python script, such as this:
34
35.. code-block::  python
36
37    from __future__ import division, print_function
38    import os,sys
39    sys.path.insert(0,'/Users/toby/software/G2/GSASII') # needed to "find" GSAS-II modules
40    import GSASIIscriptable as G2sc
41    datadir = "/Users/Scratch/"
42    gpx = G2sc.G2Project(os.path.join(datadir,'test2.gpx'))
43    gpx.histogram(0).add_back_peak(4.5,30000,5000,0)
44    pardict = {'set': {'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
45                       'Background': {'type': 'chebyschev', 'refine': True,
46                                      'peaks':[[0,True]]}}}
47    gpx.set_refinement(pardict)
48
49GSASIIscriptable can be used to setup and start sequential refinements. This script
50is used to take the single-dataset fit at the end of Step 1 of the
51`Sequential Refinement tutorial <https://subversion.xray.aps.anl.gov/pyGSAS/Tutorials/SeqRefine/SequentialTutorial.htm>`
52and turn off refinement flags, add histograms and setup the sequential fit, which is then run:
53
54.. code-block::  python
55
56    import os,sys,glob
57    sys.path.insert(0,'/Users/toby/software/G2/GSASII')
58    import GSASIIscriptable as G2sc
59    datadir = os.path.expanduser("~/Scratch/SeqTut2019Mar")
60    PathWrap = lambda fil: os.path.join(datadir,fil)
61    # load and rename project
62    gpx = G2sc.G2Project(PathWrap('7Konly.gpx'))
63    gpx.save(PathWrap('SeqRef.gpx'))
64    # turn off some variables; turn on Dijs
65    for p in gpx.phases():
66        p.set_refinements({"Cell": False})
67    gpx.phase(0).set_HAP_refinements(
68        {'Scale': False,
69         "Size": {'type':'isotropic', 'refine': False},
70         "Mustrain": {'type':'uniaxial', 'refine': False},
71         "HStrain":True,})
72    gpx.phase(1).set_HAP_refinements({'Scale': False})
73    gpx.histogram(0).clear_refinements({'Background':False,
74                     'Sample Parameters':['DisplaceX'],})
75    gpx.histogram(0).ref_back_peak(0,[])
76    gpx.phase(1).set_HAP_refinements({"HStrain":(1,1,1,0)})
77    for fil in sorted(glob.glob(PathWrap('*.fxye'))): # load in remaining fxye files
78        if '00' in fil: continue
79        gpx.add_powder_histogram(fil, PathWrap('OH_00.prm'), fmthint="GSAS powder",phases='all')
80    # copy HAP values, background, instrument params. & limits, not sample params.
81    gpx.copyHistParms(0,'all',['b','i','l'])
82    for p in gpx.phases(): p.copyHAPvalues(0,'all')
83    # setup and launch sequential fit
84    gpx.set_Controls('sequential',gpx.histograms())
85    gpx.set_Controls('cycles',10)
86    gpx.set_Controls('seqCopy',True)
87    gpx.refine() 
88
89Most functionality is provided via the objects and methods described in this section.
90
91---------------------
92:class:`G2Project`
93---------------------
94
95  All GSASIIscriptable scripts will need to create a :class:`G2Project` object
96  either for a new GSAS-II project or to read in an existing project (.gpx) file.
97  The most commonly used routines in this object are:
98
99.. tabularcolumns:: |l|p{3.5in}|
100
101==================================================    ===============================================================================================================
102method                                                Use
103==================================================    ===============================================================================================================
104:meth:`G2Project.save`                                Writes the current project to disk.
105
106:meth:`G2Project.add_powder_histogram`                Used to read in powder diffraction data into a project file.
107
108:meth:`G2Project.add_simulated_powder_histogram`      Defines a "dummy" powder diffraction data that will be simulated after a refinement step.
109
110:meth:`G2Project.add_image`                           Reads in an image into a project.
111
112:meth:`G2Project.add_phase`                           Adds a phase to a project
113
114:meth:`G2Project.histograms`                          Provides a list of histograms in the current project, as :class:`G2PwdrData` objects
115
116:meth:`G2Project.phases`                              Provides a list of phases defined in the current project, as :class:`G2Phase` objects
117
118:meth:`G2Project.images`                              Provides a list of images in the current project, as :class:`G2Image` objects
119
120:meth:`G2Project.do_refinements`                      This is passed a list of dictionaries, where each dict defines a refinement step.
121                                                      Passing a list with a single empty dict initiates a refinement with the current
122                                                      parameters and flags. A refinement dict sets up a single refinement step
123                                                      (as described in :ref:`Project_dicts`). Also see :ref:`Refinement_recipe`.
124
125:meth:`G2Project.set_refinement`                      This is passed a single dict which is used to set parameters and flags.
126                                                      These actions can be performed also in :meth:`G2Project.do_refinements`.
127==================================================    ===============================================================================================================
128
129---------------------
130:class:`G2Phase`
131---------------------
132
133  Another common object in GSASIIscriptable scripts is :class:`G2Phase`, used to encapsulate each phase in a project, with commonly used methods:
134
135.. tabularcolumns:: |l|p{3.5in}|
136
137==================================================    ===============================================================================================================
138method                                                Use
139==================================================    ===============================================================================================================
140:meth:`G2Phase.set_refinements`                       Provides a mechanism to set values and refinement flags for the phase. See :ref:`Phase_parameters_table`
141                                                      for more details. This information also can be supplied within a call to :meth:`G2Project.do_refinements`
142                                                      or :meth:`G2Project.set_refinement`.
143:meth:`G2Phase.clear_refinements`                     Unsets refinement flags for the phase.
144:meth:`G2Phase.set_HAP_refinements`                   Provides a mechanism to set values and refinement flags for parameters specific to both this phase and
145                                                      one of its histograms. See :ref:`HAP_parameters_table`. This information also can be supplied within
146                                                      a call to :meth:`G2Project.do_refinements` or :meth:`G2Project.set_refinement`.
147:meth:`G2Phase.clear_HAP_refinements`                 Clears refinement flags specific to both this phase and one of its histograms.
148:meth:`G2Phase.getHAPvalues`                          Returns values of parameters specific to both this phase and one of its histograms.
149:meth:`G2Phase.copyHAPvalues`                         Copies HAP settings between from one phase/histogram and to other histograms in same phase.
150:meth:`G2Phase.atoms`                                 Returns a list of atoms in the phase
151:meth:`G2Phase.atom`                                  Returns an atom from its label
152:meth:`G2Phase.histograms`                            Returns a list of histograms linked to the phase
153:meth:`G2Phase.get_cell`                              Returns unit cell parameters (also see :meth:`G2Phase.get_cell_and_esd`)
154:meth:`G2Phase.export_CIF`                            Writes a CIF for the phase
155:meth:`G2Phase.set_Controls`                          Set overall GSAS-II control settings such as number of cycles and to set up a sequential
156                                                      fit. (Also see :meth:`G2Phase.get_Controls` to read values.)
157==================================================    ===============================================================================================================
158
159---------------------
160:class:`G2PwdrData`
161---------------------
162
163  Another common object in GSASIIscriptable scripts is :class:`G2PwdrData`, which encapsulate each powder diffraction histogram in a project, with commonly used methods:
164
165.. tabularcolumns:: |l|p{3.5in}|
166
167==================================================    ===============================================================================================================
168method                                                Use
169==================================================    ===============================================================================================================
170:meth:`G2PwdrData.set_refinements`                    Provides a mechanism to set values and refinement flags for the powder histogram. See
171                                                      :ref:`Histogram_parameters_table` for details. 
172:meth:`G2PwdrData.clear_refinements`                  Unsets refinement flags for the the powder histogram.
173:meth:`G2PwdrData.residuals`                          Reports R-factors etc. for the the powder histogram (also see :meth:`G2PwdrData.get_wR`)
174:meth:`G2PwdrData.add_back_peak`                      Adds a background peak to the histogram. Also see :meth:`G2PwdrData.del_back_peak` and
175                                                      :meth:`G2PwdrData.ref_back_peak`.
176:meth:`G2PwdrData.fit_fixed_points`                   Fits background to the specified fixed points.
177:meth:`G2PwdrData.getdata`                            Provides access to the diffraction data associated with the histogram.
178:meth:`G2PwdrData.Export`                             Writes the diffraction data into a file
179==================================================    ===============================================================================================================
180
181---------------------
182:class:`G2Image`
183---------------------
184
185  When working with images, there will be a :class:`G2Image` object for each image (also see :meth:`G2Project.add_image`  and :meth:`G2Project.images`).
186
187.. tabularcolumns:: |l|p{3.5in}|
188
189==================================================    ===============================================================================================================
190method                                                Use
191==================================================    ===============================================================================================================
192:meth:`G2Image.Recalibrate`                           Invokes a recalibration fit starting from the current Image Controls calibration coefficients.
193:meth:`G2Image.Integrate`                             Invokes an image integration All parameters Image Controls will have previously been set.
194:meth:`G2Image.setControl`                            Set an Image Controls parameter in the current image.
195:meth:`G2Image.getControl`                            Return an Image Controls parameter in the current image.
196:meth:`G2Image.findControl`                           Get the names of Image Controls parameters.
197:meth:`G2Image.loadControls`                          Load controls from a .imctrl file (also see :meth:`G2Image.saveControls`).
198:meth:`G2Image.loadMasks`                             Load masks from a .immask file.
199:meth:`G2Image.setVary`                               Set a refinement flag for Image Controls parameter in the current image. (Also see :meth:`G2Image.getVary`)
200:meth:`G2Image.setCalibrant`                          Set a calibrant type (or show choices) for the current image.
201:meth:`G2Image.setControlFile`                        Set a image to be used as a background/dark/gain map image.
202==================================================    ===============================================================================================================
203
204----------------------
205:class:`G2AtomRecord`
206----------------------
207
208  When working with phases, :class:`G2AtomRecord` objects provide access to the contents of each atom in a phase. This provides access to "properties" that can be
209  used to get values of much of the atoms associated settings: label, type, refinement_flags, coordinates, occupancy, ranId, adp_flag, and uiso. In addition,
210  refinement_flags, occupancy and uiso can be used to set values. See the :class:`G2AtomRecord` docs and source code.
211
212.. _Refinement_dicts:
213
214=====================
215Refinement parameters
216=====================
217While scripts can be written that setup refinements by changing individual parameters
218through calls to the methods associated with objects that wrap each data tree item,
219many of these actions can be combined into fairly complex dict structures to conduct refinement
220steps. Use of these dicts is required with the :ref:`CommandlineInterface`. This section of the
221documentation describes these dicts.
222
223.. _Project_dicts:
224
225-----------------------------
226Project-level Parameter Dict
227-----------------------------
228
229As noted below (:ref:`Refinement_parameters_kinds`), there are three types of refinement parameters,
230which can be accessed individually by the objects that encapsulate individual phases and histograms
231but it will often be simplest to create a composite dictionary
232that is used at the project-level. A dict is created with keys
233"set" and "clear" that can be supplied to :meth:`G2Project.set_refinement`
234(or :meth:`G2Project.do_refinements`, see :ref:`Refinement_recipe` below) that will
235determine parameter values and will determine which parameters will be refined.
236
237The specific keys and subkeys that can be used are defined in tables
238:ref:`Histogram_parameters_table`, :ref:`Phase_parameters_table` and :ref:`HAP_parameters_table`.
239
240Note that optionally a list of histograms and/or phases can be supplied in the call to
241:meth:`G2Project.set_refinement`, but if not specified, the default is to use all defined
242phases and histograms.
243
244As an example:
245
246.. code-block::  python
247
248    pardict = {'set': { 'Limits': [0.8, 12.0],
249                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
250                       'Background': {'type': 'chebyschev', 'refine': True,
251                                      'peaks':[[0,True],[1,1,1]] }},
252              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
253    my_project.set_refinement(pardict)
254   
255.. _Refinement_recipe:
256   
257------------------------
258Refinement recipe
259------------------------
260Building on the :ref:`Project_dicts`,
261it is possible to specify a sequence of refinement actions as a list of
262these dicts and supplying this list
263as an argument to :meth:`G2Project.do_refinements`.
264
265As an example, this code performs the same actions as in the example in the section above:
266
267.. code-block::  python
268   
269    pardict = {'set': { 'Limits': [0.8, 12.0],
270                       'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
271                       'Background': {'type': 'chebyschev', 'refine': True}},
272              'clear': {'Instrument Parameters': ['U', 'V', 'W']}}
273    my_project.do_refinements([pardict])
274
275However, in addition to setting a number of parameters, this example will perform a refinement as well,
276after setting the parameters. More than one refinement can be performed by including more
277than one dict in the list.
278
279In this example, two refinement steps will be performed:
280
281.. code-block::  python
282
283    my_project.do_refinements([pardict,pardict1])
284
285
286The keys defined in the following table
287may be used in a dict supplied to :meth:`G2Project.do_refinements`. Note that keys ``histograms``
288and ``phases`` are used to limit actions to specific sets of parameters within the project.
289
290========== ============================================================================
291key         explanation
292========== ============================================================================
293set                    Specifies a dict with keys and subkeys as described in the
294                       :ref:`Refinement_parameters_fmt` section. Items listed here
295                       will be set to be refined.
296clear                  Specifies a dict, as above for set, except that parameters are
297                       cleared and thus will not be refined.
298once                   Specifies a dict as above for set, except that parameters are
299                       set for the next cycle of refinement and are cleared once the
300                       refinement step is completed.
301skip                   Normally, once parameters are processed with a set/clear/once
302                       action(s), a refinement is started. If skip is defined as True
303                       (or any other value) the refinement step is not performed.
304output                 If a file name is specified for output is will be used to save
305                       the current refinement.
306histograms             Should contain a list of histogram(s) to be used for the
307                       set/clear/once action(s) on :ref:`Histogram_parameters_table` or
308                       :ref:`HAP_parameters_table`. Note that this will be
309                       ignored for :ref:`Phase_parameters_table`. Histograms may be
310                       specified as a list of strings [('PWDR ...'),...], indices
311                       [0,1,2] or as list of objects [hist1, hist2].
312phases                 Should contain a list of phase(s) to be used for the
313                       set/clear/once action(s) on :ref:`Phase_parameters_table` or
314                       :ref:`HAP_parameters_table`. Note that this will be
315                       ignored for :ref:`Histogram_parameters_table`.
316                       Phases may be specified as a list of strings
317                       [('Phase name'),...], indices [0,1,2] or as list of objects
318                       [phase0, phase2].
319call                   Specifies a function to call after a refinement is completed.
320                       The value supplied can be the object (typically a function)
321                       that will be called or a string that will evaluate (in the
322                       namespace inside :meth:`G2Project.iter_refinements` where
323                       ``self`` references the project.)
324                       Nothing is called if this is not specified.
325callargs               Provides a list of arguments that will be passed to the function
326                       in call (if any). If call is defined and callargs is not, the
327                       current <tt>G2Project</tt> is passed as a single argument.
328========== ============================================================================
329
330An example that performs a series of refinement steps follows:
331
332.. code-block::  python
333
334    reflist = [
335            {"set": { "Limits": { "low": 0.7 },
336                      "Background": { "no. coeffs": 3,
337                                      "refine": True }}},
338            {"set": { "LeBail": True,
339                      "Cell": True }},
340            {"set": { "Sample Parameters": ["DisplaceX"]}},
341            {"set": { "Instrument Parameters": ["U", "V", "W", "X", "Y"]}},
342            {"set": { "Mustrain": { "type": "uniaxial",
343                                    "refine": "equatorial",
344                                    "direction": [0, 0, 1]}}},
345            {"set": { "Mustrain": { "type": "uniaxial",
346                                    "refine": "axial"}}},
347            {"clear": { "LeBail": True},
348             "set": { "Atoms": { "Mn": "X" }}},
349            {"set": { "Atoms": { "O1": "X", "O2": "X" }}},]
350    my_project.do_refinements(reflist)
351   
352
353In this example, a separate refinement step will be performed for each dict in the list. The keyword
354"skip" can be used to specify a dict that should not include a refinement.
355Note that in the second from last refinement step, parameters are both set and cleared.
356   
357.. _Refinement_parameters_kinds:
358
359----------------------------
360Refinement parameter types
361----------------------------
362
363Note that parameters and refinement flags used in GSAS-II fall into three classes:
364
365    * **Histogram**: There will be a set of these for each dataset loaded into a
366      project file. The parameters available depend on the type of histogram
367      (Bragg-Brentano, Single-Crystal, TOF,...). Typical Histogram parameters
368      include the overall scale factor, background, instrument and sample parameters;
369      see the :ref:`Histogram_parameters_table` table for a list of the histogram
370      parameters where access has been provided.
371     
372    * **Phase**: There will be a set of these for each phase loaded into a
373      project file. While some parameters are found in all types of phases,
374      others are only found in certain types (modulated, magnetic, protein...).
375      Typical phase parameters include unit cell lengths and atomic positions; see the
376      :ref:`Phase_parameters_table` table for a list of the phase     
377      parameters where access has been provided.
378     
379    * **Histogram-and-phase** (HAP): There is a set of these for every histogram
380      that is associated with each phase, so that if there are ``N`` phases and ``M``
381      histograms, there can be ``N*M`` total sets of "HAP" parameters sets (fewer if all
382      histograms are not linked to all phases.) Typical HAP parameters include the
383      phase fractions, sample microstrain and crystallite size broadening terms,
384      hydrostatic strain perturbations of the unit cell and preferred orientation
385      values.
386      See the :ref:`HAP_parameters_table` table for the HAP parameters where access has
387      been provided.
388
389.. _Refinement_parameters_fmt:
390
391=================================
392Specifying Refinement Parameters
393=================================
394
395Refinement parameter values and flags to turn refinement on and off are specified within dictionaries,
396where the details of these dicts are organized depends on the
397type of parameter (see :ref:`Refinement_parameters_kinds`), with a different set
398of keys (as described below) for each of the three types of parameters.
399
400.. _Histogram_parameters_table:
401
402--------------------
403Histogram parameters
404--------------------
405
406This table describes the dictionaries supplied to :func:`G2PwdrData.set_refinements`
407and :func:`G2PwdrData.clear_refinements`. As an example,
408
409.. code-block::  python
410
411   hist.set_refinements({"Background": {"no.coeffs": 3, "refine": True},
412                         "Sample Parameters": ["Scale"],
413                         "Limits": [10000, 40000]})
414
415With :meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
416``set``, ``clear``, or ``once``. Values will be set for all histograms, unless the ``histograms``
417key is used to define specific histograms. As an example:
418
419.. code-block::  python
420
421  gsas_proj.do_refinements([
422      {'set': {
423          'Background': {'no.coeffs': 3, 'refine': True},
424          'Sample Parameters': ['Scale'],
425          'Limits': [10000, 40000]},
426      'histograms': [1,2]}
427                            ])
428
429Note that below in the Instrument Parameters section,
430related profile parameters (such as U and V) are grouped together but
431separated by commas to save space in the table.
432
433.. tabularcolumns:: |l|l|p{3.5in}|
434
435===================== ====================  =================================================
436key                   subkey                explanation
437===================== ====================  =================================================
438Limits                                      The range of 2-theta (degrees) or TOF (in
439                                            microsec) range of values to use. Can
440                                            be either a dictionary of 'low' and/or 'high',
441                                            or a list of 2 items [low, high]
442\                     low                   Sets the low limit
443\                     high                  Sets the high limit
444
445Sample Parameters                           Should be provided as a **list** of subkeys
446                                            to set or clear, e.g. ['DisplaceX', 'Scale']
447\                     Absorption
448\                     Contrast
449\                     DisplaceX             Sample displacement along the X direction
450\                     DisplaceY             Sample displacement along the Y direction
451\                     Scale                 Histogram Scale factor
452
453Background                                  Sample background. Value will be a dict or
454                                            a boolean. If True or False, the refine
455                                            parameter for background is set to that.
456                                            Note that background peaks are not handled
457                                            via this; see
458                                            :meth:`G2PwdrData.ref_back_peak` instead.
459                                            When value is a dict,
460                                            supply any of the following keys:
461\                     type                  The background model, e.g. 'chebyschev'
462\                     refine                The value of the refine flag, boolean
463\                     no. coeffs            Number of coefficients to use, integer
464\                     coeffs                List of floats, literal values for background
465\                     FixedPoints           List of (2-theta, intensity) values for fixed points
466\                     fit fixed points      If True, triggers a fit to the fixed points to
467                                            be calculated. It is calculated when this key is
468                                            detected, regardless of calls to refine.
469                      peaks                 Specifies a set of flags for refining
470                                            background peaks as a nested list. There may
471                                            be an item for each defined background peak
472                                            (or fewer) and each item is a list with the flag
473                                            values for pos,int,sig & gam (fewer than 4 values
474                                            are allowed).
475
476Instrument Parameters                       As in Sample Paramters, provide as a **list** of
477                                            subkeys to
478                                            set or clear, e.g. ['X', 'Y', 'Zero', 'SH/L']
479\                     U, V, W               Gaussian peak profile terms
480\                     X, Y, Z               Lorentzian peak profile terms
481\                     alpha, beta-0,        TOF profile terms
482                      beta-1, beta-q,
483\                     sig-0, sig-1,         TOF profile terms
484                      sig-2, sig-q
485\                     difA, difB, difC      TOF Calibration constants
486\                     Zero                  Zero shift
487\                     SH/L                  Finger-Cox-Jephcoat low-angle peak asymmetry
488\                     Polariz.              Polarization parameter
489\                     Lam                   Lambda, the incident wavelength
490===================== ====================  =================================================
491
492.. _Phase_parameters_table:
493
494----------------
495Phase parameters
496----------------
497
498This table describes the dictionaries supplied to :func:`G2Phase.set_refinements`
499and :func:`G2Phase.clear_refinements`. With :meth:`G2Project.do_refinements`,
500these parameters should be placed inside a dict with a key
501``set``, ``clear``, or ``once``. Values will be set for all phases, unless the ``phases``
502key is used to define specific phase(s).
503
504
505.. tabularcolumns:: |l|p{4.5in}|
506
507======= ==========================================================
508key                   explanation
509======= ==========================================================
510Cell                  Whether or not to refine the unit cell.
511Atoms                 Dictionary of atoms and refinement flags.
512                      Each key should be an atom label, e.g.
513                      'O3', 'Mn5', and each value should be
514                      a string defining what values to refine.
515                      Values can be any combination of 'F'
516                      for fractional occupancy, 'X' for position,
517                      and 'U' for Debye-Waller factor
518LeBail                Enables LeBail intensity extraction.
519======= ==========================================================
520
521
522.. _HAP_parameters_table:
523
524
525Histogram-and-phase parameters
526------------------------------
527
528This table describes the dictionaries supplied to :func:`G2Phase.set_HAP_refinements`
529and :func:`G2Phase.clear_HAP_refinements`. When supplied to
530:meth:`G2Project.do_refinements`, these parameters should be placed inside a dict with a key
531``set``, ``clear``, or ``once``. Values will be set for all histograms used in each phase,
532unless the ``histograms`` and ``phases`` keys are used to define specific phases and histograms.
533
534.. tabularcolumns:: |l|l|p{3.5in}|
535
536=============  ==========  ============================================================
537key             subkey                 explanation
538=============  ==========  ============================================================
539Babinet                                Should be a **list** of the following
540                                       subkeys. If not, assumes both
541                                       BabA and BabU
542\               BabA
543\               BabU
544Extinction                             Boolean, True to refine.
545HStrain                                Boolean or list/tuple, True to refine all
546                                       appropriate D\ :sub:`ij` terms or False
547                                       to not refine any. If a list/tuple, will
548                                       be a set of True & False values for each
549                                       D\ :sub:`ij` term; number of items must
550                                       match number of terms.
551Mustrain
552\               type                   Mustrain model. One of 'isotropic',
553                                       'uniaxial', or 'generalized'. **Should always
554                                       be included when Mustrain is used.**
555\              direction               For uniaxial only. A list of three
556                                       integers,
557                                       the [hkl] direction of the axis.
558\               refine                 Usually boolean, set to True to refine.
559                                       or False to clear.
560                                       For uniaxial model, can specify a value
561                                       of 'axial' or 'equatorial' to set that flag
562                                       to True or a single
563                                       boolean sets both axial and equatorial.
564Size                                   
565\               type                   Size broadening model. One of 'isotropic',
566                                       'uniaxial', or 'ellipsoid'. **Should always
567                                       be specified when Size is used.**
568\              direction               For uniaxial only. A list of three
569                                       integers,
570                                       the [hkl] direction of the axis.
571\               refine                 Boolean, True to refine.
572\               value                  float, size value in microns
573Pref.Ori.                              Boolean, True to refine
574Show                                   Boolean, True to refine
575Use                                    Boolean, True to refine
576Scale                                  Phase fraction; Boolean, True to refine
577=============  ==========  ============================================================
578
579------------------------
580Histogram/Phase objects
581------------------------
582Each phase and powder histogram in a :class:`G2Project` object has an associated
583object. Parameters within each individual object can be turned on and off by calling
584:meth:`G2PwdrData.set_refinements` or :meth:`G2PwdrData.clear_refinements`
585for histogram parameters;
586:meth:`G2Phase.set_refinements` or :meth:`G2Phase.clear_refinements`
587for phase parameters; and :meth:`G2Phase.set_HAP_refinements` or
588:meth:`G2Phase.clear_HAP_refinements`. As an example, if some_histogram is a histogram object (of type :class:`G2PwdrData`), use this to set parameters in that histogram:
589
590.. code-block::  python
591
592    params = { 'Limits': [0.8, 12.0],
593               'Sample Parameters': ['Absorption', 'Contrast', 'DisplaceX'],
594               'Background': {'type': 'chebyschev', 'refine': True}}
595    some_histogram.set_refinements(params)
596
597Likewise to turn refinement flags on, use code such as this:
598
599.. code-block::  python
600
601    params = { 'Instrument Parameters': ['U', 'V', 'W']}
602    some_histogram.set_refinements(params)
603
604and to turn these refinement flags, off use this (Note that the
605``.clear_refinements()`` methods will usually will turn off refinement even
606if a refinement parameter is set in the dict to True.):
607
608.. code-block::  python
609
610    params = { 'Instrument Parameters': ['U', 'V', 'W']}
611    some_histogram.clear_refinements(params)
612
613For phase parameters, use code such as this:
614   
615.. code-block::  python
616
617    params = { 'LeBail': True, 'Cell': True,
618               'Atoms': { 'Mn1': 'X',
619                          'O3': 'XU',
620                          'V4': 'FXU'}}
621    some_histogram.set_refinements(params)
622
623
624and here is an example for HAP parameters:
625
626.. code-block::  python
627
628    params = { 'Babinet': 'BabA',
629               'Extinction': True,
630               'Mustrain': { 'type': 'uniaxial',
631                             'direction': [0, 0, 1],
632                             'refine': True}}
633    some_phase.set_HAP_refinements(params)
634
635Note that the parameters must match the object type and method (phase vs. histogram vs. HAP).
636
637.. _CommandlineInterface:
638
639=======================================
640GSASIIscriptable Command-line Interface
641=======================================
642
643The routines described above are intended to be called from a Python script, but an
644alternate way to access some of the same functionality is to
645invoke the ``GSASIIscriptable.py`` script from
646the command line usually from within a shell script or batch file. This
647will usually be done with a command such as::
648
649       python <path/>GSASIIscriptable.py <subcommand> <file.gpx> <options>
650
651The following subcommands are defined:
652
653        * create, see :func:`create`
654        * add, see :func:`add`
655        * dump, see :func:`dump`
656        * refine, see :func:`refine`
657        * export, :func:`export`
658        * browse, see :func:`IPyBrowse`
659       
660Run::
661
662   python GSASIIscriptable.py --help
663
664to show the available subcommands, and inspect each subcommand with
665`python GSASIIscriptable.py <subcommand> --help` or see the documentation for each of the above routines.
666
667.. _JsonFormat:
668
669-------------------------
670Parameters in JSON files
671-------------------------
672
673The refine command requires two inputs: an existing GSAS-II project (.gpx) file and
674a JSON format file
675(see `Introducing JSON <http://json.org/>`_) that contains a single dict.
676This dict may have two keys:
677
678refinements:
679  This defines the a set of refinement steps in a JSON representation of a
680  :ref:`Refinement_recipe` list.
681
682code:
683  This optionally defines Python code that will be executed after the project is loaded,
684  but before the refinement is started. This can be used to execute Python code to change
685  parameters that are not accessible via a :ref:`Refinement_recipe` dict (note that the
686  project object is accessed with variable ``proj``) or to define code that will be called
687  later (see key ``call`` in the :ref:`Refinement_recipe` section.)
688   
689JSON website: `Introducing JSON <http://json.org/>`_.
690
691.. _API:
692
693============================================================
694API: Complete Documentation
695============================================================
696
697The large number of classes and modules in this module are described below.
698A script will have one or more G2Project objects using :class:`G2Project` and then
699perform actions such as adding a histogram (method :meth:`G2Project.add_powder_histogram`),
700adding a phase (method :meth:`G2Project.add_phase`),
701or setting parameters and performing a refinement
702(method :meth:`G2Project.do_refinements`).
703
704To change settings within histograms, images and phases, one usually needs to use
705methods inside :class:`G2PwdrData`, :class:`G2Image` or :class:`G2Phase`.
706
707"""
708from __future__ import division, print_function
709import argparse
710import os.path as ospath
711import datetime as dt
712import sys
713import platform
714if '2' in platform.python_version_tuple()[0]:
715    import cPickle
716    strtypes = (str,unicode)
717else:
718    import pickle as cPickle
719    strtypes = (str,bytes)
720import imp
721import copy
722import os
723import random as ran
724
725import numpy.ma as ma
726import scipy.interpolate as si
727import numpy as np
728import scipy as sp
729
730import GSASIIpath
731GSASIIpath.SetBinaryPath(True)  # for now, this is needed before some of these modules can be imported
732import GSASIIobj as G2obj
733import GSASIIpwd as G2pwd
734import GSASIIstrMain as G2strMain
735#import GSASIIIO as G2IO
736import GSASIIstrIO as G2strIO
737import GSASIIspc as G2spc
738import GSASIIElem as G2elem
739import GSASIIfiles as G2fil
740import GSASIIimage as G2img
741
742# Delay imports to not slow down small scripts that don't need them
743Readers = {'Pwdr':[], 'Phase':[], 'Image':[]}
744'''Readers by reader type'''
745exportersByExtension = {}
746'''Specifies the list of extensions that are supported for Powder data export'''
747npsind = lambda x: np.sin(x*np.pi/180.)
748
749def LoadG2fil():
750    """Setup GSAS-II importers. Delay importing this module, it is slow"""
751    if len(Readers['Pwdr']) > 0: return
752    # initialize imports
753    Readers['Pwdr'] = G2fil.LoadImportRoutines("pwd", "Powder_Data")
754    Readers['Phase'] = G2fil.LoadImportRoutines("phase", "Phase")
755    Readers['Image'] = G2fil.LoadImportRoutines("img", "Image")
756
757    # initialize exports
758    for obj in G2fil.LoadExportRoutines(None):
759        try:
760            obj.Writer
761        except AttributeError:
762            continue
763        for typ in obj.exporttype:
764            if typ not in exportersByExtension:
765                exportersByExtension[typ] = {obj.extension:obj}
766            else:
767                exportersByExtension[typ][obj.extension] = obj
768
769def LoadDictFromProjFile(ProjFile):
770    '''Read a GSAS-II project file and load items to dictionary
771   
772    :param str ProjFile: GSAS-II project (name.gpx) full file name
773    :returns: Project,nameList, where
774
775      * Project (dict) is a representation of gpx file following the GSAS-II tree structure
776        for each item: key = tree name (e.g. 'Controls','Restraints',etc.), data is dict
777        data dict = {'data':item data whch may be list, dict or None,'subitems':subdata (if any)}
778      * nameList (list) has names of main tree entries & subentries used to reconstruct project file
779
780    Example for fap.gpx::
781
782      Project = {                 #NB:dict order is not tree order
783        'Phases':{'data':None,'fap':{phase dict}},
784        'PWDR FAP.XRA Bank 1':{'data':[histogram data list],'Comments':comments,'Limits':limits, etc},
785        'Rigid bodies':{'data': {rigid body dict}},
786        'Covariance':{'data':{covariance data dict}},
787        'Controls':{'data':{controls data dict}},
788        'Notebook':{'data':[notebook list]},
789        'Restraints':{'data':{restraint data dict}},
790        'Constraints':{'data':{constraint data dict}}]
791        }
792      nameList = [                #NB: reproduces tree order
793        ['Notebook',],
794        ['Controls',],
795        ['Covariance',],
796        ['Constraints',],
797        ['Restraints',],
798        ['Rigid bodies',],
799        ['PWDR FAP.XRA Bank 1',
800             'Comments',
801             'Limits',
802             'Background',
803             'Instrument Parameters',
804             'Sample Parameters',
805             'Peak List',
806             'Index Peak List',
807             'Unit Cells List',
808             'Reflection Lists'],
809        ['Phases', 'fap']
810        ]
811    '''
812    # Let IOError be thrown if file does not exist
813    if not ospath.exists(ProjFile):
814        print ('\n*** Error attempt to open project file that does not exist: \n    {}'.
815                   format(ProjFile))
816        raise IOError('GPX file {} does not exist'.format(ProjFile))
817    try:
818        Project, nameList = G2strIO.GetFullGPX(ProjFile)
819    except Exception as msg:
820        raise IOError(msg)
821    return Project,nameList
822
823def SaveDictToProjFile(Project,nameList,ProjFile):
824    '''Save a GSAS-II project file from dictionary/nameList created by LoadDictFromProjFile
825
826    :param dict Project: representation of gpx file following the GSAS-II
827        tree structure as described for LoadDictFromProjFile
828    :param list nameList: names of main tree entries & subentries used to reconstruct project file
829    :param str ProjFile: full file name for output project.gpx file (including extension)
830    '''
831    file = open(ProjFile,'wb')
832    try:
833        for name in nameList:
834            data = []
835            item = Project[name[0]]
836            data.append([name[0],item['data']])
837            for item2 in name[1:]:
838                data.append([item2,item[item2]])
839            cPickle.dump(data,file,1)
840    finally:
841        file.close()
842    print('gpx file saved as %s'%ProjFile)
843
844# def ImportPowder(reader,filename):
845#     '''Use a reader to import a powder diffraction data file
846
847#     :param str reader: a scriptable reader
848#     :param str filename: full name of powder data file; can be "multi-Bank" data
849
850#     :returns: list rdlist: list of reader objects containing powder data, one for each
851#         "Bank" of data encountered in file. Items in reader object of interest are:
852
853#           * rd.comments: list of str: comments found on powder file
854#           * rd.dnames: list of str: data nammes suitable for use in GSASII data tree NB: duplicated in all rd entries in rdlist
855#           * rd.powderdata: list of numpy arrays: pos,int,wt,zeros,zeros,zeros as needed for a PWDR entry in  GSASII data tree.
856#     '''
857#     rdfile,rdpath,descr = imp.find_module(reader)
858#     rdclass = imp.load_module(reader,rdfile,rdpath,descr)
859#     rd = rdclass.GSAS_ReaderClass()
860#     if not rd.scriptable:
861#         print(u'**** ERROR: '+reader+u' is not a scriptable reader')
862#         return None
863#     rdlist = []
864#     if rd.ContentsValidator(filename):
865#         repeat = True
866#         rdbuffer = {} # create temporary storage for file reader
867#         block = 0
868#         while repeat: # loop if the reader asks for another pass on the file
869#             block += 1
870#             repeat = False
871#             rd.objname = ospath.basename(filename)
872#             flag = rd.Reader(filename,None,buffer=rdbuffer,blocknum=block,)
873#             if flag:
874#                 rdlist.append(copy.deepcopy(rd)) # save the result before it is written over
875#                 if rd.repeat:
876#                     repeat = True
877#         return rdlist
878#     print(rd.errors)
879#     return None
880
881def SetDefaultDData(dType,histoName,NShkl=0,NDij=0):
882    '''Create an initial Histogram dictionary
883
884    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
885    '''
886    if dType in ['SXC','SNC']:
887        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
888            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
889            'Extinction':['Lorentzian','None', {'Tbar':0.1,'Cos2TM':0.955,
890            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}],
891            'Flack':[0.0,False]}
892    elif dType == 'SNT':
893        return {'Histogram':histoName,'Show':False,'Scale':[1.0,True],
894            'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]},
895            'Extinction':['Lorentzian','None', {
896            'Eg':[1.e-10,False],'Es':[1.e-10,False],'Ep':[1.e-10,False]}]}
897    elif 'P' in dType:
898        return {'Histogram':histoName,'Show':False,'Scale':[1.0,False],
899            'Pref.Ori.':['MD',1.0,False,[0,0,1],0,{},[],0.1],
900            'Size':['isotropic',[1.,1.,1.],[False,False,False],[0,0,1],
901                [1.,1.,1.,0.,0.,0.],6*[False,]],
902            'Mustrain':['isotropic',[1000.0,1000.0,1.0],[False,False,False],[0,0,1],
903                NShkl*[0.01,],NShkl*[False,]],
904            'HStrain':[NDij*[0.0,],NDij*[False,]],
905            'Extinction':[0.0,False],'Babinet':{'BabA':[0.0,False],'BabU':[0.0,False]}}
906
907
908def PreSetup(data):
909    '''Create part of an initial (empty) phase dictionary
910
911    from GSASIIphsGUI.py, near end of UpdatePhaseData
912
913    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
914    '''
915    if 'RBModels' not in data:
916        data['RBModels'] = {}
917    if 'MCSA' not in data:
918        data['MCSA'] = {'Models':[{'Type':'MD','Coef':[1.0,False,[.8,1.2],],'axis':[0,0,1]}],'Results':[],'AtInfo':{}}
919    if 'dict' in str(type(data['MCSA']['Results'])):
920        data['MCSA']['Results'] = []
921    if 'Modulated' not in data['General']:
922        data['General']['Modulated'] = False
923#    if 'modulated' in data['General']['Type']:
924#        data['General']['Modulated'] = True
925#        data['General']['Type'] = 'nuclear'
926
927
928def SetupGeneral(data, dirname):
929    """Helps initialize phase data.
930
931    From GSASIIphsGui.py, function of the same name. Minor changes for imports etc.
932
933    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
934    """
935    mapDefault = {'MapType':'','RefList':'','Resolution':0.5,'Show bonds':True,
936                'rho':[],'rhoMax':0.,'mapSize':10.0,'cutOff':50.,'Flip':False}
937    generalData = data['General']
938    atomData = data['Atoms']
939    generalData['AtomTypes'] = []
940    generalData['Isotopes'] = {}
941
942    if 'Isotope' not in generalData:
943        generalData['Isotope'] = {}
944    if 'Data plot type' not in generalData:
945        generalData['Data plot type'] = 'Mustrain'
946    if 'POhkl' not in generalData:
947        generalData['POhkl'] = [0,0,1]
948    if 'Map' not in generalData:
949        generalData['Map'] = mapDefault.copy()
950    if 'Flip' not in generalData:
951        generalData['Flip'] = {'RefList':'','Resolution':0.5,'Norm element':'None',
952            'k-factor':0.1,'k-Max':20.,}
953    if 'testHKL' not in generalData['Flip']:
954        generalData['Flip']['testHKL'] = [[0,0,2],[2,0,0],[1,1,1],[0,2,0],[1,2,3]]
955    if 'doPawley' not in generalData:
956        generalData['doPawley'] = False     #ToDo: change to ''
957    if 'Pawley dmin' not in generalData:
958        generalData['Pawley dmin'] = 1.0
959    if 'Pawley neg wt' not in generalData:
960        generalData['Pawley neg wt'] = 0.0
961    if 'Algolrithm' in generalData.get('MCSA controls',{}) or \
962        'MCSA controls' not in generalData:
963        generalData['MCSA controls'] = {'Data source':'','Annealing':[50.,0.001,50],
964        'dmin':2.0,'Algorithm':'log','Jump coeff':[0.95,0.5],'boltzmann':1.0,
965        'fast parms':[1.0,1.0,1.0],'log slope':0.9,'Cycles':1,'Results':[],'newDmin':True}
966    if 'AtomPtrs' not in generalData:
967        generalData['AtomPtrs'] = [3,1,7,9]
968        if generalData['Type'] == 'macromolecular':
969            generalData['AtomPtrs'] = [6,4,10,12]
970        elif generalData['Type'] == 'magnetic':
971            generalData['AtomPtrs'] = [3,1,10,12]
972    if generalData['Modulated']:
973        generalData['Type'] = 'nuclear'
974        if 'Super' not in generalData:
975            generalData['Super'] = 1
976            generalData['SuperVec'] = [[0,0,.1],False,4]
977            generalData['SSGData'] = {}
978        if '4DmapData' not in generalData:
979            generalData['4DmapData'] = mapDefault.copy()
980            generalData['4DmapData'].update({'MapType':'Fobs'})
981    if 'Modulated' not in generalData:
982        generalData['Modulated'] = False
983    if 'HydIds' not in generalData:
984        generalData['HydIds'] = {}
985    cx,ct,cs,cia = generalData['AtomPtrs']
986    generalData['NoAtoms'] = {}
987    generalData['BondRadii'] = []
988    generalData['AngleRadii'] = []
989    generalData['vdWRadii'] = []
990    generalData['AtomMass'] = []
991    generalData['Color'] = []
992    if generalData['Type'] == 'magnetic':
993        generalData['MagDmin'] = generalData.get('MagDmin',1.0)
994        landeg = generalData.get('Lande g',[])
995    generalData['Mydir'] = dirname
996    badList = {}
997    for iat,atom in enumerate(atomData):
998        atom[ct] = atom[ct].lower().capitalize()              #force to standard form
999        if generalData['AtomTypes'].count(atom[ct]):
1000            generalData['NoAtoms'][atom[ct]] += atom[cx+3]*float(atom[cs+1])
1001        elif atom[ct] != 'UNK':
1002            Info = G2elem.GetAtomInfo(atom[ct])
1003            if not Info:
1004                if atom[ct] not in badList:
1005                    badList[atom[ct]] = 0
1006                badList[atom[ct]] += 1
1007                atom[ct] = 'UNK'
1008                continue
1009            atom[ct] = Info['Symbol'] # N.B. symbol might be changed by GetAtomInfo
1010            generalData['AtomTypes'].append(atom[ct])
1011            generalData['Z'] = Info['Z']
1012            generalData['Isotopes'][atom[ct]] = Info['Isotopes']
1013            generalData['BondRadii'].append(Info['Drad'])
1014            generalData['AngleRadii'].append(Info['Arad'])
1015            generalData['vdWRadii'].append(Info['Vdrad'])
1016            if atom[ct] in generalData['Isotope']:
1017                if generalData['Isotope'][atom[ct]] not in generalData['Isotopes'][atom[ct]]:
1018                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
1019                    generalData['Isotope'][atom[ct]] = isotope
1020                generalData['AtomMass'].append(Info['Isotopes'][generalData['Isotope'][atom[ct]]]['Mass'])
1021            else:
1022                generalData['Isotope'][atom[ct]] = 'Nat. Abund.'
1023                if 'Nat. Abund.' not in generalData['Isotopes'][atom[ct]]:
1024                    isotope = list(generalData['Isotopes'][atom[ct]].keys())[-1]
1025                    generalData['Isotope'][atom[ct]] = isotope
1026                generalData['AtomMass'].append(Info['Mass'])
1027            generalData['NoAtoms'][atom[ct]] = atom[cx+3]*float(atom[cs+1])
1028            generalData['Color'].append(Info['Color'])
1029            if generalData['Type'] == 'magnetic':
1030                if len(landeg) < len(generalData['AtomTypes']):
1031                    landeg.append(2.0)
1032    if generalData['Type'] == 'magnetic':
1033        generalData['Lande g'] = landeg[:len(generalData['AtomTypes'])]
1034
1035    if badList:
1036        msg = 'Warning: element symbol(s) not found:'
1037        for key in badList:
1038            msg += '\n\t' + key
1039            if badList[key] > 1:
1040                msg += ' (' + str(badList[key]) + ' times)'
1041        raise G2ScriptException("Phase error:\n" + msg)
1042        # wx.MessageBox(msg,caption='Element symbol error')
1043    F000X = 0.
1044    F000N = 0.
1045    for i,elem in enumerate(generalData['AtomTypes']):
1046        F000X += generalData['NoAtoms'][elem]*generalData['Z']
1047        isotope = generalData['Isotope'][elem]
1048        F000N += generalData['NoAtoms'][elem]*generalData['Isotopes'][elem][isotope]['SL'][0]
1049    generalData['F000X'] = F000X
1050    generalData['F000N'] = F000N
1051    import GSASIImath as G2mth
1052    generalData['Mass'] = G2mth.getMass(generalData)
1053
1054
1055def make_empty_project(author=None, filename=None):
1056    """Creates an dictionary in the style of GSASIIscriptable, for an empty
1057    project.
1058
1059    If no author name or filename is supplied, 'no name' and
1060    <current dir>/test_output.gpx are used , respectively.
1061
1062    Returns: project dictionary, name list
1063
1064    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1065    """
1066    if not filename:
1067        filename = 'test_output.gpx'
1068    filename = os.path.abspath(filename)
1069    gsasii_version = str(GSASIIpath.GetVersionNumber())
1070    LoadG2fil()
1071    try:
1072        import matplotlib as mpl
1073        python_library_versions = G2fil.get_python_versions([mpl, np, sp])
1074    except ImportError:
1075        python_library_versions = G2fil.get_python_versions([np, sp])
1076
1077    controls_data = dict(G2obj.DefaultControls)
1078    controls_data['LastSavedAs'] = filename
1079    controls_data['LastSavedUsing'] = gsasii_version
1080    controls_data['PythonVersions'] = python_library_versions
1081    if author:
1082        controls_data['Author'] = author
1083
1084    output = {'Constraints': {'data': {'HAP': [], 'Hist': [], 'Phase': [],
1085                                       'Global': []}},
1086              'Controls': {'data': controls_data},
1087              u'Covariance': {'data': {}},
1088              u'Notebook': {'data': ['']},
1089              u'Restraints': {'data': {}},
1090              u'Rigid bodies': {'data': {'RBIds': {'Residue': [], 'Vector': []},
1091                                'Residue': {'AtInfo': {}},
1092                                'Vector':  {'AtInfo': {}}}}}
1093
1094    names = [[u'Notebook'], [u'Controls'], [u'Covariance'],
1095             [u'Constraints'], [u'Restraints'], [u'Rigid bodies']]
1096
1097    return output, names
1098
1099
1100class G2ImportException(Exception):
1101    pass
1102
1103class G2ScriptException(Exception):
1104    pass
1105
1106def import_generic(filename, readerlist, fmthint=None, bank=None):
1107    """Attempt to import a filename, using a list of reader objects.
1108
1109    Returns the first reader object which worked."""
1110    # Translated from OnImportGeneric method in GSASII.py
1111    primaryReaders, secondaryReaders = [], []
1112    for reader in readerlist:
1113        if fmthint is not None and fmthint not in reader.formatName: continue
1114        flag = reader.ExtensionValidator(filename)
1115        if flag is None:
1116            secondaryReaders.append(reader)
1117        elif flag:
1118            primaryReaders.append(reader)
1119    if not secondaryReaders and not primaryReaders:
1120        raise G2ImportException("Could not read file: ", filename)
1121
1122    with open(filename, 'Ur') as fp:
1123        rd_list = []
1124
1125        for rd in primaryReaders + secondaryReaders:
1126            # Initialize reader
1127            rd.selections = []
1128            if bank is None:
1129                rd.selections = []
1130            else:
1131                rd.selections = [bank-1]
1132            rd.dnames = []
1133            rd.ReInitialize()
1134            # Rewind file
1135            rd.errors = ""
1136            if not rd.ContentsValidator(filename):
1137                # Report error
1138                print("Warning: File {} has a validation error, continuing".format(filename))
1139            if len(rd.selections) > 1:
1140                raise G2ImportException("File {} has {} banks. Specify which bank to read with databank param."
1141                                .format(filename,len(rd.selections)))
1142
1143            block = 0
1144            rdbuffer = {}
1145            repeat = True
1146            while repeat:
1147                repeat = False
1148                block += 1
1149                rd.objname = os.path.basename(filename)
1150                try:
1151                    flag = rd.Reader(filename,buffer=rdbuffer, blocknum=block)
1152                except:
1153                    flag = False
1154                if flag:
1155                    # Omitting image loading special cases
1156                    rd.readfilename = filename
1157                    rd_list.append(copy.deepcopy(rd))
1158                    repeat = rd.repeat
1159                else:
1160                    if GSASIIpath.GetConfigValue('debug'): print("DBG_{} Reader failed to read {}".format(rd.formatName,filename))
1161            if rd_list:
1162                if rd.warnings:
1163                    print("Read warning by", rd.formatName, "reader:",
1164                          rd.warnings, file=sys.stderr)
1165                elif bank is None:
1166                    print("{} read by Reader {}"
1167                              .format(filename,rd.formatName))
1168                else:
1169                    print("{} block # {} read by Reader {}"
1170                              .format(filename,bank,rd.formatName))
1171                return rd_list
1172    raise G2ImportException("No reader could read file: " + filename)
1173
1174
1175def load_iprms(instfile, reader, bank=None):
1176    """Loads instrument parameters from a file, and edits the
1177    given reader.
1178
1179    Returns a 2-tuple of (Iparm1, Iparm2) parameters
1180    """
1181    LoadG2fil()
1182    ext = os.path.splitext(instfile)[1]
1183
1184    if ext.lower() == '.instprm':
1185        # New GSAS File, load appropriate bank
1186        with open(instfile) as f:
1187            lines = f.readlines()
1188        if bank is None: 
1189            bank = reader.powderentry[2] 
1190        numbanks = reader.numbanks
1191        iparms = G2fil.ReadPowderInstprm(lines, bank, numbanks, reader)
1192        reader.instfile = instfile
1193        reader.instmsg = '{} (G2 fmt) bank {}'.format(instfile,bank)
1194        return iparms
1195    elif ext.lower() not in ('.prm', '.inst', '.ins'):
1196        raise ValueError('Expected .prm file, found: ', instfile)
1197
1198    # It's an old GSAS file, load appropriately
1199    Iparm = {}
1200    with open(instfile, 'Ur') as fp:
1201        for line in fp:
1202            if '#' in line:
1203                continue
1204            Iparm[line[:12]] = line[12:-1]
1205    ibanks = int(Iparm.get('INS   BANK  ', '1').strip())
1206    if bank is not None:
1207        # pull out requested bank # bank from the data, and change the bank to 1
1208        Iparm,IparmC = {},Iparm
1209        for key in IparmC:
1210            if 'INS' not in key[:3]: continue   #skip around rubbish lines in some old iparm
1211            if key[4:6] == "  ":
1212                Iparm[key] = IparmC[key]
1213            elif int(key[4:6].strip()) == bank:
1214                Iparm[key[:4]+' 1'+key[6:]] = IparmC[key]           
1215        reader.instbank = bank
1216    elif ibanks == 1:
1217        reader.instbank = 1
1218    else: 
1219        raise G2ImportException("Instrument parameter file has {} banks, select one with instbank param."
1220                                    .format(ibanks))
1221    reader.powderentry[2] = 1
1222    reader.instfile = instfile
1223    reader.instmsg = '{} bank {}'.format(instfile,reader.instbank)
1224    return G2fil.SetPowderInstParms(Iparm, reader)
1225
1226def load_pwd_from_reader(reader, instprm, existingnames=[],bank=None):
1227    """Loads powder data from a reader object, and assembles it into a GSASII data tree.
1228
1229    :returns: (name, tree) - 2-tuple of the histogram name (str), and data
1230
1231    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1232    """
1233    HistName = 'PWDR ' + G2obj.StripUnicode(reader.idstring, '_')
1234    HistName = G2obj.MakeUniqueLabel(HistName, existingnames)
1235
1236    try:
1237        Iparm1, Iparm2 = instprm
1238    except ValueError:
1239        Iparm1, Iparm2 = load_iprms(instprm, reader, bank=bank)
1240        print('Instrument parameters read:',reader.instmsg)
1241    Ymin = np.min(reader.powderdata[1])
1242    Ymax = np.max(reader.powderdata[1])
1243    valuesdict = {'wtFactor': 1.0,
1244                  'Dummy': False,
1245                  'ranId': ran.randint(0, sys.maxsize),
1246                  'Offset': [0.0, 0.0], 'delOffset': 0.02*Ymax,
1247                  'refOffset': -0.1*Ymax, 'refDelt': 0.1*Ymax,
1248                  'Yminmax': [Ymin, Ymax]}
1249    reader.Sample['ranId'] = valuesdict['ranId']
1250    if 'T' in Iparm1['Type'][0]:
1251        if not reader.clockWd and reader.GSAS:
1252            reader.powderdata[0] *= 100.        #put back the CW centideg correction
1253
1254    # Ending keys:
1255    # [u'Reflection Lists',
1256    #  u'Limits',
1257    #  'data',
1258    #  u'Index Peak List',
1259    #  u'Comments',
1260    #  u'Unit Cells List',
1261    #  u'Sample Parameters',
1262    #  u'Peak List',
1263    #  u'Background',
1264    #  u'Instrument Parameters']
1265    Tmin = np.min(reader.powderdata[0])
1266    Tmax = np.max(reader.powderdata[0])
1267
1268    default_background = [['chebyschev', False, 3, 1.0, 0.0, 0.0],
1269                          {'nDebye': 0, 'debyeTerms': [], 'nPeaks': 0, 'peaksList': []}]
1270
1271    output_dict = {u'Reflection Lists': {},
1272                   u'Limits': reader.pwdparms.get('Limits', [(Tmin, Tmax), [Tmin, Tmax]]),
1273                   u'data': [valuesdict, reader.powderdata, HistName],
1274                   u'Index Peak List': [[], []],
1275                   u'Comments': reader.comments,
1276                   u'Unit Cells List': [],
1277                   u'Sample Parameters': reader.Sample,
1278                   u'Peak List': {'peaks': [], 'sigDict': {}},
1279                   u'Background': reader.pwdparms.get('Background', default_background),
1280                   u'Instrument Parameters': [Iparm1, Iparm2],
1281                   }
1282
1283    names = [u'Comments',
1284             u'Limits',
1285             u'Background',
1286             u'Instrument Parameters',
1287             u'Sample Parameters',
1288             u'Peak List',
1289             u'Index Peak List',
1290             u'Unit Cells List',
1291             u'Reflection Lists']
1292
1293    # TODO controls?? GSASII.py:1664-7
1294
1295    return HistName, [HistName] + names, output_dict
1296
1297def _deep_copy_into(from_, into):
1298    """Helper function for reloading .gpx file. See G2Project.reload()
1299
1300    :author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1301    """
1302    if isinstance(from_, dict) and isinstance(into, dict):
1303        combined_keys = set(from_.keys()).union(into.keys())
1304        for key in combined_keys:
1305            if key in from_ and key in into:
1306                both_dicts = (isinstance(from_[key], dict)
1307                              and isinstance(into[key], dict))
1308                both_lists = (isinstance(from_[key], list)
1309                              and isinstance(into[key], list))
1310                if both_dicts or both_lists:
1311                    _deep_copy_into(from_[key], into[key])
1312                else:
1313                    into[key] = from_[key]
1314            elif key in from_:
1315                into[key] = from_[key]
1316            else:  # key in into
1317                del into[key]
1318    elif isinstance(from_, list) and isinstance(into, list):
1319        if len(from_) == len(into):
1320            for i in range(len(from_)):
1321                both_dicts = (isinstance(from_[i], dict)
1322                              and isinstance(into[i], dict))
1323                both_lists = (isinstance(from_[i], list)
1324                              and isinstance(into[i], list))
1325                if both_dicts or both_lists:
1326                    _deep_copy_into(from_[i], into[i])
1327                else:
1328                    into[i] = from_[i]
1329        else:
1330            into[:] = from_
1331
1332def _getCorrImage(ImageReaderlist,proj,imageRef):
1333    '''Gets image & applies dark, background & flat background corrections.
1334    based on :func:`GSASIIimgGUI.GetImageZ`. Expected to be for internal
1335    use only.
1336
1337    :param list ImageReaderlist: list of Reader objects for images
1338    :param object ImageReaderlist: list of Reader objects for images
1339    :param imageRef: A reference to the desired image. Either the Image
1340      tree name (str), the image's index (int) or
1341      a image object (:class:`G2Image`)
1342
1343    :return: array sumImg: corrected image for background/dark/flat back
1344    '''
1345    ImgObj = proj.image(imageRef)
1346    Controls = ImgObj.data['Image Controls']
1347    formatName = Controls.get('formatName','')
1348    imagefile = ImgObj.data['data'][1]
1349    ImageTag = None # fix this for multiimage files
1350    sumImg = G2fil.RereadImageData(ImageReaderlist,imagefile,ImageTag=ImageTag,FormatName=formatName)
1351    if sumImg is None:
1352        return []
1353    darkImg = False
1354    if 'dark image' in Controls:
1355        darkImg,darkScale = Controls['dark image']
1356        if darkImg:
1357            dImgObj = proj.image(darkImg)
1358            formatName = dImgObj.data['Image Controls'].get('formatName','')
1359            imagefile = dImgObj.data['data'][1]
1360            ImageTag = None # fix this for multiimage files
1361            darkImage = G2fil.RereadImageData(ImageReaderlist,imagefile,ImageTag=ImageTag,FormatName=formatName)
1362            if darkImg is None:
1363                raise Exception('Error reading dark image {}'.format(imagefile))
1364            sumImg += np.array(darkImage*darkScale,dtype='int32')
1365    if 'background image' in Controls:
1366        backImg,backScale = Controls['background image']           
1367        if backImg:     #ignores any transmission effect in the background image
1368            bImgObj = proj.image(backImg)
1369            formatName = bImgObj.data['Image Controls'].get('formatName','')
1370            imagefile = bImgObj.data['data'][1]
1371            ImageTag = None # fix this for multiimage files
1372            backImage = G2fil.RereadImageData(ImageReaderlist,imagefile,ImageTag=ImageTag,FormatName=formatName)
1373            if backImage is None:
1374                raise Exception('Error reading background image {}'.format(imagefile))
1375            if darkImg:
1376                backImage += np.array(darkImage*darkScale/backScale,dtype='int32')
1377            else:
1378                sumImg += np.array(backImage*backScale,dtype='int32')
1379    if 'Gain map' in Controls:
1380        gainMap = Controls['Gain map']
1381        if gainMap:
1382            gImgObj = proj.image(gainMap)
1383            formatName = gImgObj.data['Image Controls'].get('formatName','')
1384            imagefile = gImgObj.data['data'][1]
1385            ImageTag = None # fix this for multiimage files
1386            GMimage = G2fil.RereadImageData(ImageReaderlist,imagefile,ImageTag=ImageTag,FormatName=formatName)
1387            if GMimage is None:
1388                raise Exception('Error reading Gain map image {}'.format(imagefile))
1389            sumImg = sumImg*GMimage/1000
1390    sumImg -= int(Controls.get('Flat Bkg',0))
1391    Imax = np.max(sumImg)
1392    Controls['range'] = [(0,Imax),[0,Imax]]
1393    return np.asarray(sumImg,dtype='int32')
1394
1395class G2ObjectWrapper(object):
1396    """Base class for all GSAS-II object wrappers.
1397
1398    The underlying GSAS-II format can be accessed as `wrapper.data`. A number
1399    of overrides are implemented so that the wrapper behaves like a dictionary.
1400
1401    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
1402    """
1403    def __init__(self, datadict):
1404        self.data = datadict
1405
1406    def __getitem__(self, key):
1407        return self.data[key]
1408
1409    def __setitem__(self, key, value):
1410        self.data[key] = value
1411
1412    def __contains__(self, key):
1413        return key in self.data
1414
1415    def get(self, k, d=None):
1416        return self.data.get(k, d)
1417
1418    def keys(self):
1419        return self.data.keys()
1420
1421    def values(self):
1422        return self.data.values()
1423
1424    def items(self):
1425        return self.data.items()
1426
1427
1428class G2Project(G2ObjectWrapper):   
1429    """Represents an entire GSAS-II project.
1430
1431    :param str gpxfile: Existing .gpx file to be loaded. If nonexistent,
1432            creates an empty project.
1433    :param str author: Author's name (not yet implemented)
1434    :param str newgpx: The filename the project should be saved to in
1435            the future. If both newgpx and gpxfile are present, the project is
1436            loaded from the gpxfile, then when saved will be written to newgpx.
1437    :param str filename: Name to be used to save the project. Has same function as
1438            parameter newgpx (do not use both gpxfile and filename). Use of newgpx
1439            is preferred over filename.
1440
1441    There are two ways to initialize this object:
1442
1443    >>> # Load an existing project file
1444    >>> proj = G2Project('filename.gpx')
1445   
1446    >>> # Create a new project
1447    >>> proj = G2Project(newgpx='new_file.gpx')
1448   
1449    Histograms can be accessed easily.
1450
1451    >>> # By name
1452    >>> hist = proj.histogram('PWDR my-histogram-name')
1453   
1454    >>> # Or by index
1455    >>> hist = proj.histogram(0)
1456    >>> assert hist.id == 0
1457   
1458    >>> # Or by random id
1459    >>> assert hist == proj.histogram(hist.ranId)
1460
1461    Phases can be accessed the same way.
1462
1463    >>> phase = proj.phase('name of phase')
1464
1465    New data can also be loaded via :meth:`~G2Project.add_phase` and
1466    :meth:`~G2Project.add_powder_histogram`.
1467
1468    >>> hist = proj.add_powder_histogram('some_data_file.chi',
1469                                         'instrument_parameters.prm')
1470    >>> phase = proj.add_phase('my_phase.cif', histograms=[hist])
1471
1472    Parameters for Rietveld refinement can be turned on and off as well.
1473    See :meth:`~G2Project.set_refinement`, :meth:`~G2Project.clear_refinements`,
1474    :meth:`~G2Project.iter_refinements`, :meth:`~G2Project.do_refinements`.
1475    """
1476    def __init__(self, gpxfile=None, author=None, filename=None, newgpx=None):
1477        if filename is not None and newgpx is not None:
1478            raise G2ScriptException('Do not use filename and newgpx together')
1479        elif newgpx is not None:
1480            filename = newgpx
1481        if gpxfile is None:
1482            filename = os.path.abspath(os.path.expanduser(filename))
1483            self.filename = filename
1484            self.data, self.names = make_empty_project(author=author, filename=filename)
1485        elif isinstance(gpxfile, str): # TODO: create replacement for isinstance that checks if path exists
1486                                       # filename is valid, etc.
1487            if isinstance(filename, str): 
1488                self.filename = os.path.abspath(os.path.expanduser(filename)) # both are defined
1489            else: 
1490                self.filename = os.path.abspath(os.path.expanduser(gpxfile))
1491            # TODO set author
1492            self.data, self.names = LoadDictFromProjFile(gpxfile)
1493            self.update_ids()
1494        else:
1495            raise ValueError("Not sure what to do with gpxfile")
1496
1497    @classmethod
1498    def from_dict_and_names(cls, gpxdict, names, filename=None):
1499        """Creates a :class:`G2Project` directly from
1500        a dictionary and a list of names. If in doubt, do not use this.
1501
1502        :returns: a :class:`G2Project`
1503        """
1504        out = cls()
1505        if filename:
1506            filename = os.path.abspath(os.path.expanduser(filename))
1507            out.filename = filename
1508            gpxdict['Controls']['data']['LastSavedAs'] = filename
1509        else:
1510            try:
1511                out.filename = gpxdict['Controls']['data']['LastSavedAs']
1512            except KeyError:
1513                out.filename = None
1514        out.data = gpxdict
1515        out.names = names
1516
1517    def save(self, filename=None):
1518        """Saves the project, either to the current filename, or to a new file.
1519
1520        Updates self.filename if a new filename provided"""
1521        # TODO update LastSavedUsing ?
1522        if filename:
1523            filename = os.path.abspath(os.path.expanduser(filename))
1524            self.data['Controls']['data']['LastSavedAs'] = filename
1525            self.filename = filename
1526        elif not self.filename:
1527            raise AttributeError("No file name to save to")
1528        SaveDictToProjFile(self.data, self.names, self.filename)
1529
1530    def add_powder_histogram(self, datafile, iparams, phases=[], fmthint=None,
1531                                 databank=None, instbank=None):
1532        """Loads a powder data histogram into the project.
1533
1534        Automatically checks for an instrument parameter file, or one can be
1535        provided. Note that in unix fashion, "~" can be used to indicate the
1536        home directory (e.g. ~/G2data/data.fxye).
1537
1538        :param str datafile: The powder data file to read, a filename.
1539        :param str iparams: The instrument parameters file, a filename.
1540        :param list phases: A list of phases to link to the new histogram,
1541           phases can be references by object, name, rId or number.
1542           Alternately, use 'all' to link to all phases in the project.
1543        :param str fmthint: If specified, only importers where the format name
1544          (reader.formatName, as shown in Import menu) contains the
1545          supplied string will be tried as importers. If not specified, all
1546          importers consistent with the file extension will be tried
1547          (equivalent to "guess format" in menu).
1548        :param int databank: Specifies a dataset number to read, if file contains
1549          more than set of data. This should be 1 to read the first bank in
1550          the file (etc.) regardless of the number on the Bank line, etc.
1551          Default is None which means there should only be one dataset in the
1552          file.
1553        :param int instbank: Specifies an instrument parameter set to read, if
1554          the instrument parameter file contains more than set of parameters.
1555          This will match the INS # in an GSAS type file so it will typically
1556          be 1 to read the first parameter set in the file (etc.)
1557          Default is None which means there should only be one parameter set
1558          in the file.
1559
1560        :returns: A :class:`G2PwdrData` object representing
1561            the histogram
1562        """
1563        LoadG2fil()
1564        datafile = os.path.abspath(os.path.expanduser(datafile))
1565        iparams = os.path.abspath(os.path.expanduser(iparams))
1566        pwdrreaders = import_generic(datafile, Readers['Pwdr'],fmthint=fmthint,bank=databank)
1567        histname, new_names, pwdrdata = load_pwd_from_reader(
1568                                          pwdrreaders[0], iparams,
1569                                          [h.name for h in self.histograms()],bank=instbank)
1570        if histname in self.data:
1571            print("Warning - redefining histogram", histname)
1572        elif self.names[-1][0] == 'Phases':
1573            self.names.insert(-1, new_names)
1574        else:
1575            self.names.append(new_names)
1576        self.data[histname] = pwdrdata
1577        self.update_ids()
1578
1579        if phases == 'all':
1580            phases = self.phases()
1581        for phase in phases:
1582            phase = self.phase(phase)
1583            self.link_histogram_phase(histname, phase)
1584
1585        return self.histogram(histname)
1586
1587    def add_simulated_powder_histogram(self, histname, iparams, Tmin, Tmax, Tstep,
1588                                       wavelength=None, scale=None, phases=[]):
1589        """Loads a powder data histogram into the project.
1590
1591        Requires an instrument parameter file.
1592        Note that in unix fashion, "~" can be used to indicate the
1593        home directory (e.g. ~/G2data/data.prm). The instrument parameter file
1594        will determine if the histogram is x-ray, CW neutron, TOF, etc. as well
1595        as the instrument type.
1596
1597        :param str histname: A name for the histogram to be created.
1598        :param str iparams: The instrument parameters file, a filename.
1599        :param float Tmin: Minimum 2theta or TOF (ms) for dataset to be simulated
1600        :param float Tmax: Maximum 2theta or TOF (ms) for dataset to be simulated
1601        :param float Tstep: Step size in 2theta or TOF (ms) for dataset to be simulated       
1602        :param float wavelength: Wavelength for CW instruments, overriding the value
1603           in the instrument parameters file if specified.
1604        :param float scale: Histogram scale factor which multiplies the pattern. Note that
1605           simulated noise is added to the pattern, so that if the maximum intensity is
1606           small, the noise will mask the computed pattern. The scale
1607           needs to be a large number for CW neutrons.
1608           The default, None, provides a scale of 1 for x-rays and TOF; 10,000 for CW neutrons.
1609        :param list phases: Phases to link to the new histogram. Use proj.phases() to link to
1610           all defined phases.
1611
1612        :returns: A :class:`G2PwdrData` object representing the histogram
1613        """
1614        LoadG2fil()
1615        iparams = os.path.abspath(os.path.expanduser(iparams))
1616        if not os.path.exists(iparams):
1617            raise G2ScriptException("File does not exist:"+iparams)
1618        rd = G2obj.ImportPowderData( # Initialize a base class reader
1619            extensionlist=tuple(),
1620            strictExtension=False,
1621            formatName = 'Simulate dataset',
1622            longFormatName = 'Compute a simulated pattern')
1623        rd.powderentry[0] = '' # no filename
1624        rd.powderentry[2] = 1 # only one bank
1625        rd.comments.append('This is a dummy dataset for powder pattern simulation')
1626        rd.idstring = histname
1627        #Iparm1, Iparm2 = load_iprms(iparams, rd)
1628        if Tmax < Tmin:
1629            Tmin,Tmax = Tmax,Tmin
1630        Tstep = abs(Tstep)
1631        if 'TOF' in rd.idstring:
1632                N = (np.log(Tmax)-np.log(Tmin))/Tstep
1633                x = np.exp((np.arange(0,N))*Tstep+np.log(Tmin*1000.))
1634                N = len(x)
1635        else:           
1636                N = int((Tmax-Tmin)/Tstep)+1
1637                x = np.linspace(Tmin,Tmax,N,True)
1638                N = len(x)
1639        if N < 3:
1640            raise G2ScriptException("Error: Range is too small or step is too large, <3 points")
1641        rd.powderdata = [
1642            np.array(x), # x-axis values
1643            np.zeros_like(x), # powder pattern intensities
1644            np.ones_like(x), # 1/sig(intensity)^2 values (weights)
1645            np.zeros_like(x), # calc. intensities (zero)
1646            np.zeros_like(x), # calc. background (zero)
1647            np.zeros_like(x), # obs-calc profiles
1648            ]
1649        Tmin = rd.powderdata[0][0]
1650        Tmax = rd.powderdata[0][-1]
1651        histname, new_names, pwdrdata = load_pwd_from_reader(rd, iparams,
1652                                                            [h.name for h in self.histograms()])
1653        if histname in self.data:
1654            print("Warning - redefining histogram", histname)
1655        elif self.names[-1][0] == 'Phases':
1656            self.names.insert(-1, new_names)
1657        else:
1658            self.names.append(new_names)
1659        if scale is not None:
1660            pwdrdata['Sample Parameters']['Scale'][0] = scale
1661        elif pwdrdata['Instrument Parameters'][0]['Type'][0].startswith('PNC'):
1662            pwdrdata['Sample Parameters']['Scale'][0] = 10000.
1663        self.data[histname] = pwdrdata
1664        self.update_ids()
1665
1666        for phase in phases:
1667            phase = self.phase(phase)
1668            self.link_histogram_phase(histname, phase)
1669
1670        return self.histogram(histname)
1671   
1672    def add_phase(self, phasefile, phasename=None, histograms=[], fmthint=None):
1673        """Loads a phase into the project from a .cif file
1674
1675        :param str phasefile: The CIF file from which to import the phase.
1676        :param str phasename: The name of the new phase, or None for the default
1677        :param list histograms: The names of the histograms to associate with
1678            this phase. Use proj.histograms() to add to all histograms.
1679        :param str fmthint: If specified, only importers where the format name
1680          (reader.formatName, as shown in Import menu) contains the
1681          supplied string will be tried as importers. If not specified, all
1682          importers consistent with the file extension will be tried
1683          (equivalent to "guess format" in menu).
1684
1685        :returns: A :class:`G2Phase` object representing the
1686            new phase.
1687        """
1688        LoadG2fil()
1689        histograms = [self.histogram(h).name for h in histograms]
1690        phasefile = os.path.abspath(os.path.expanduser(phasefile))
1691
1692        # TODO handle multiple phases in a file
1693        phasereaders = import_generic(phasefile, Readers['Phase'], fmthint=fmthint)
1694        phasereader = phasereaders[0]
1695       
1696        phasename = phasename or phasereader.Phase['General']['Name']
1697        phaseNameList = [p.name for p in self.phases()]
1698        phasename = G2obj.MakeUniqueLabel(phasename, phaseNameList)
1699        phasereader.Phase['General']['Name'] = phasename
1700
1701        if 'Phases' not in self.data:
1702            self.data[u'Phases'] = { 'data': None }
1703        assert phasename not in self.data['Phases'], "phase names should be unique"
1704        self.data['Phases'][phasename] = phasereader.Phase
1705
1706        if phasereader.Constraints:
1707            Constraints = self.data['Constraints']
1708            for i in phasereader.Constraints:
1709                if isinstance(i, dict):
1710                    if '_Explain' not in Constraints:
1711                        Constraints['_Explain'] = {}
1712                    Constraints['_Explain'].update(i)
1713                else:
1714                    Constraints['Phase'].append(i)
1715
1716        data = self.data['Phases'][phasename]
1717        generalData = data['General']
1718        SGData = generalData['SGData']
1719        NShkl = len(G2spc.MustrainNames(SGData))
1720        NDij = len(G2spc.HStrainNames(SGData))
1721        Super = generalData.get('Super', 0)
1722        if Super:
1723            SuperVec = np.array(generalData['SuperVec'][0])
1724        else:
1725            SuperVec = []
1726        UseList = data['Histograms']
1727
1728        for hist in histograms:
1729            self.link_histogram_phase(hist, phasename)
1730
1731        for obj in self.names:
1732            if obj[0] == 'Phases':
1733                phasenames = obj
1734                break
1735        else:
1736            phasenames = [u'Phases']
1737            self.names.append(phasenames)
1738        phasenames.append(phasename)
1739
1740        # TODO should it be self.filename, not phasefile?
1741        SetupGeneral(data, os.path.dirname(phasefile))
1742        self.index_ids()
1743
1744        self.update_ids()
1745        return self.phase(phasename)
1746
1747    def link_histogram_phase(self, histogram, phase):
1748        """Associates a given histogram and phase.
1749
1750        .. seealso::
1751
1752            :meth:`~G2Project.histogram`
1753            :meth:`~G2Project.phase`"""
1754        hist = self.histogram(histogram)
1755        phase = self.phase(phase)
1756
1757        generalData = phase['General']
1758
1759        if hist.name.startswith('HKLF '):
1760            raise NotImplementedError("HKLF not yet supported")
1761        elif hist.name.startswith('PWDR '):
1762            hist['Reflection Lists'][generalData['Name']] = {}
1763            UseList = phase['Histograms']
1764            SGData = generalData['SGData']
1765            NShkl = len(G2spc.MustrainNames(SGData))
1766            NDij = len(G2spc.HStrainNames(SGData))
1767            UseList[hist.name] = SetDefaultDData('PWDR', hist.name, NShkl=NShkl, NDij=NDij)
1768            UseList[hist.name]['hId'] = hist.id
1769            for key, val in [('Use', True), ('LeBail', False),
1770                             ('newLeBail', True),
1771                             ('Babinet', {'BabA': [0.0, False],
1772                                          'BabU': [0.0, False]})]:
1773                if key not in UseList[hist.name]:
1774                    UseList[hist.name][key] = val
1775        else:
1776            raise RuntimeError("Unexpected histogram" + hist.name)
1777
1778    def reload(self):
1779        """Reload self from self.filename"""
1780        data, names = LoadDictFromProjFile(self.filename)
1781        self.names = names
1782        # Need to deep copy the new data file data into the current tree,
1783        # so that any existing G2Phase, or G2PwdrData objects will still be
1784        # valid
1785        _deep_copy_into(from_=data, into=self.data)
1786
1787    def refine(self, newfile=None, printFile=None, makeBack=False):
1788        '''Invoke a refinement for the project. The project is written to
1789        the currently selected gpx file and then either a single or sequential refinement
1790        is performed depending on the setting of 'Seq Data' in Controls
1791        (set in :meth:`get_Controls`).
1792        '''
1793        seqSetting = self.data['Controls']['data'].get('Seq Data',[])
1794        if not seqSetting:
1795            self.index_ids()    # index_ids will automatically save the project
1796            # TODO: migrate to RefineCore G2strMain does not properly use printFile
1797            # G2strMain.RefineCore(Controls,Histograms,Phases,restraintDict,rigidbodyDict,parmDict,varyList,
1798            #      calcControls,pawleyLookup,ifPrint,printFile,dlg)
1799            G2strMain.Refine(self.filename, makeBack=makeBack)
1800        else:
1801            self._seqrefine()
1802        self.reload() # get file from GPX
1803
1804    def _seqrefine(self):
1805        '''Perform a sequential refinement.
1806        '''
1807
1808        self.data['Controls']['data']['ShowCell'] = True
1809        # add to tree item to project, if not present
1810        if 'Sequential results' not in self.data:
1811            self.data['Sequential results'] = {'data':{}}
1812            self.names.append(['Sequential results'])
1813        self.index_ids()    # index_ids will automatically save the project
1814        #GSASIIpath.IPyBreak_base()
1815
1816        # check that constraints are OK
1817        errmsg, warnmsg = G2strIO.ReadCheckConstraints(self.filename)
1818        if errmsg:
1819            print('Refinement error',errmsg)
1820            raise Exception('Constraint error')
1821        if warnmsg:
1822            print(u'Conflict between refinment flag settings and constraints:\n'+
1823                  warnmsg+u'\nRefinement not possible')
1824            raise Exception('Constraint error')
1825        OK,Msg = G2strMain.SeqRefine(self.filename,None)
1826
1827    def histogram(self, histname):
1828        """Returns the histogram named histname, or None if it does not exist.
1829
1830        :param histname: The name of the histogram (str), or ranId or index.
1831        :returns: A :class:`G2PwdrData` object, or None if
1832            the histogram does not exist
1833
1834        .. seealso::
1835            :meth:`~G2Project.histograms`
1836            :meth:`~G2Project.phase`
1837            :meth:`~G2Project.phases`
1838        """
1839        if isinstance(histname, G2PwdrData):
1840            if histname.proj == self:
1841                return histname
1842            else:
1843                raise Exception('Histogram object (G2PwdrData) is not in current project')
1844        if histname in self.data:
1845            return G2PwdrData(self.data[histname], self, histname)
1846        try:
1847            # see if histname is an id or ranId
1848            histname = int(histname)
1849        except ValueError:
1850            return
1851
1852        for histogram in self.histograms():
1853            if histogram.id == histname or histogram.ranId == histname:
1854                return histogram
1855
1856    def histograms(self, typ=None):
1857        """Return a list of all histograms, as :class:`G2PwdrData` objects
1858
1859        For now this only finds Powder/Single Xtal histograms, since that is all that is
1860        currently implemented in this module.
1861
1862        :param ste typ: The prefix (type) the histogram such as 'PWDR '. If None
1863          (the default) all known histograms types are found.
1864        :returns: a list of objects
1865
1866        .. seealso::
1867            :meth:`~G2Project.histogram`
1868            :meth:`~G2Project.phase`
1869            :meth:`~G2Project.phases`
1870        """
1871        output = []
1872        # loop through each tree entry. If it is more than one level (more than one item in the
1873        # list of names). then it must be a histogram, unless labeled Phases or Restraints
1874        if typ is None:
1875            for obj in self.names:
1876                if obj[0].startswith('PWDR ') or obj[0].startswith('HKLF '): 
1877                    output.append(self.histogram(obj[0]))
1878        else:
1879            for obj in self.names:
1880                if len(obj) > 1 and obj[0].startswith(typ): 
1881                    output.append(self.histogram(obj[0]))
1882        return output
1883
1884    def phase(self, phasename):
1885        """
1886        Gives an object representing the specified phase in this project.
1887
1888        :param str phasename: A reference to the desired phase. Either the phase
1889            name (str), the phase's ranId, the phase's index (both int) or
1890            a phase object (:class:`G2Phase`)
1891        :returns: A :class:`G2Phase` object
1892        :raises: KeyError
1893
1894        .. seealso::
1895            :meth:`~G2Project.histograms`
1896            :meth:`~G2Project.phase`
1897            :meth:`~G2Project.phases`
1898            """
1899        if isinstance(phasename, G2Phase):
1900            if phasename.proj == self:
1901                return phasename
1902        phases = self.data['Phases']
1903        if phasename in phases:
1904            return G2Phase(phases[phasename], phasename, self)
1905
1906        try:
1907            # phasename should be phase index or ranId
1908            phasename = int(phasename)
1909        except ValueError:
1910            return
1911
1912        for phase in self.phases():
1913            if phase.id == phasename or phase.ranId == phasename:
1914                return phase
1915
1916    def phases(self):
1917        """
1918        Returns a list of all the phases in the project.
1919
1920        :returns: A list of :class:`G2Phase` objects
1921
1922        .. seealso::
1923            :meth:`~G2Project.histogram`
1924            :meth:`~G2Project.histograms`
1925            :meth:`~G2Project.phase`
1926            """
1927        for obj in self.names:
1928            if obj[0] == 'Phases':
1929                return [self.phase(p) for p in obj[1:]]
1930        return []
1931
1932    def _images(self):
1933        """Returns a list of all the phases in the project.
1934        """
1935        return [i[0] for i in self.names if i[0].startswith('IMG ')]
1936   
1937    def image(self, imageRef):
1938        """
1939        Gives an object representing the specified image in this project.
1940
1941        :param str imageRef: A reference to the desired image. Either the Image
1942          tree name (str), the image's index (int) or
1943          a image object (:class:`G2Image`)
1944        :returns: A :class:`G2Image` object
1945        :raises: KeyError
1946
1947        .. seealso::
1948            :meth:`~G2Project.images`
1949        """
1950        if isinstance(imageRef, G2Image):
1951            if imageRef.proj == self:
1952                return imageRef
1953            else:
1954                raise Exception("Image {} not in current selected project".format(imageRef.name))
1955        if imageRef in self._images():
1956            return G2Image(self.data[imageRef], imageRef, self)
1957
1958        try:
1959            # imageRef should be an index
1960            num = int(imageRef)
1961            imageRef = self._images()[num] 
1962            return G2Image(self.data[imageRef], imageRef, self)
1963        except ValueError:
1964            raise Exception("imageRef {} not an object, name or image index in current selected project"
1965                                .format(imageRef))
1966        except IndexError:
1967            raise Exception("imageRef {} out of range (max={}) in current selected project"
1968                                .format(imageRef,len(self._images())-1))
1969           
1970    def images(self):
1971        """
1972        Returns a list of all the images in the project.
1973
1974        :returns: A list of :class:`G2Image` objects
1975        """
1976        return [G2Image(self.data[i],i,self) for i in self._images()]
1977   
1978    def update_ids(self):
1979        """Makes sure all phases and histograms have proper hId and pId"""
1980        # Translated from GetUsedHistogramsAndPhasesfromTree,
1981        #   GSASIIdataGUI.py:4107
1982        for i, h in enumerate(self.histograms()):
1983            h.id = i
1984        for i, p in enumerate(self.phases()):
1985            p.id = i
1986
1987    def do_refinements(self, refinements, histogram='all', phase='all',
1988                       outputnames=None, makeBack=False):
1989        """Conducts one or a series of refinements according to the
1990           input provided in parameter refinements. This is a wrapper
1991           around :meth:`iter_refinements`
1992
1993        :param list refinements: A list of dictionaries specifiying changes to be made to
1994            parameters before refinements are conducted.
1995            See the :ref:`Refinement_recipe` section for how this is defined.
1996        :param str histogram: Name of histogram for refinements to be applied
1997            to, or 'all'; note that this can be overridden for each refinement
1998            step via a "histograms" entry in the dict.
1999        :param str phase: Name of phase for refinements to be applied to, or
2000            'all'; note that this can be overridden for each refinement
2001            step via a "phases" entry in the dict.
2002        :param list outputnames: Provides a list of project (.gpx) file names
2003            to use for each refinement step (specifying None skips the save step).
2004            See :meth:`save`.
2005            Note that this can be overridden using an "output" entry in the dict.
2006        :param bool makeBack: determines if a backup ).bckX.gpx) file is made
2007            before a refinement is performed. The default is False.
2008           
2009        To perform a single refinement without changing any parameters, use this
2010        call:
2011
2012        .. code-block::  python
2013       
2014            my_project.do_refinements([])
2015        """
2016       
2017        for proj in self.iter_refinements(refinements, histogram, phase,
2018                                          outputnames, makeBack):
2019            pass
2020        return self
2021
2022    def iter_refinements(self, refinements, histogram='all', phase='all',
2023                         outputnames=None, makeBack=False):
2024        """Conducts a series of refinements, iteratively. Stops after every
2025        refinement and yields this project, to allow error checking or
2026        logging of intermediate results. Parameter use is the same as for
2027        :meth:`do_refinements` (which calls this method).
2028
2029        >>> def checked_refinements(proj):
2030        ...     for p in proj.iter_refinements(refs):
2031        ...         # Track intermediate results
2032        ...         log(p.histogram('0').residuals)
2033        ...         log(p.phase('0').get_cell())
2034        ...         # Check if parameter diverged, nonsense answer, or whatever
2035        ...         if is_something_wrong(p):
2036        ...             raise Exception("I need a human!")
2037
2038           
2039        """
2040        if outputnames:
2041            if len(refinements) != len(outputnames):
2042                raise ValueError("Should have same number of outputs to"
2043                                 "refinements")
2044        else:
2045            outputnames = [None for r in refinements]
2046
2047        for output, refinedict in zip(outputnames, refinements):
2048            if 'histograms' in refinedict:
2049                hist = refinedict['histograms']
2050            else:
2051                hist = histogram
2052            if 'phases' in refinedict:
2053                ph = refinedict['phases']
2054            else:
2055                ph = phase
2056            if 'output' in refinedict:
2057                output = refinedict['output']
2058            self.set_refinement(refinedict, hist, ph)
2059            # Handle 'once' args - refinements that are disabled after this
2060            # refinement
2061            if 'once' in refinedict:
2062                temp = {'set': refinedict['once']}
2063                self.set_refinement(temp, hist, ph)
2064
2065            if output:
2066                self.save(output)
2067
2068            if 'skip' not in refinedict:
2069                self.refine(makeBack=makeBack)
2070            yield self
2071
2072            # Handle 'once' args - refinements that are disabled after this
2073            # refinement
2074            if 'once' in refinedict:
2075                temp = {'clear': refinedict['once']}
2076                self.set_refinement(temp, hist, ph)
2077            if 'call' in refinedict:
2078                fxn = refinedict['call']
2079                if callable(fxn):
2080                    fxn(*refinedict.get('callargs',[self]))
2081                elif callable(eval(fxn)):
2082                    eval(fxn)(*refinedict.get('callargs',[self]))
2083                else:
2084                    raise G2ScriptException("Error: call value {} is not callable".format(fxn))
2085
2086    def set_refinement(self, refinement, histogram='all', phase='all'):
2087        """Apply specified refinements to a given histogram(s) or phase(s).
2088
2089        :param dict refinement: The refinements to be conducted
2090        :param histogram: Specifies either 'all' (default), a single histogram or
2091          a list of histograms. Histograms may be specified as histogram objects
2092          (see :class:`G2PwdrData`), the histogram name (str) or the index number (int)
2093          of the histogram in the project, numbered starting from 0.
2094          Omitting the parameter or the string 'all' indicates that parameters in
2095          all histograms should be set.
2096        :param phase: Specifies either 'all' (default), a single phase or
2097          a list of phases. Phases may be specified as phase objects
2098          (see :class:`G2Phase`), the phase name (str) or the index number (int)
2099          of the phase in the project, numbered starting from 0.
2100          Omitting the parameter or the string 'all' indicates that parameters in
2101          all phases should be set.
2102
2103        Note that refinement parameters are categorized as one of three types:
2104
2105        1. Histogram parameters
2106        2. Phase parameters
2107        3. Histogram-and-Phase (HAP) parameters
2108       
2109        .. seealso::
2110            :meth:`G2PwdrData.set_refinements`
2111            :meth:`G2PwdrData.clear_refinements`
2112            :meth:`G2Phase.set_refinements`
2113            :meth:`G2Phase.clear_refinements`
2114            :meth:`G2Phase.set_HAP_refinements`
2115            :meth:`G2Phase.clear_HAP_refinements`"""
2116
2117        if histogram == 'all':
2118            hists = self.histograms()
2119        elif isinstance(histogram, list) or isinstance(histogram, tuple):
2120            hists = []
2121            for h in histogram:
2122                if isinstance(h, str) or isinstance(h, int):
2123                    hists.append(self.histogram(h))
2124                else:
2125                    hists.append(h)
2126        elif isinstance(histogram, str) or isinstance(histogram, int):
2127            hists = [self.histogram(histogram)]
2128        else:
2129            hists = [histogram]
2130
2131        if phase == 'all':
2132            phases = self.phases()
2133        elif isinstance(phase, list) or isinstance(phase, tuple):
2134            phases = []
2135            for ph in phase:
2136                if isinstance(ph, str) or isinstance(ph, int):
2137                    phases.append(self.phase(ph))
2138                else:
2139                    phases.append(ph)
2140        elif isinstance(phase, str) or isinstance(phase, int):
2141            phases = [self.phase(phase)]
2142        else:
2143            phases = [phase]
2144
2145        pwdr_set = {}
2146        phase_set = {}
2147        hap_set = {}
2148        for key, val in refinement.get('set', {}).items():
2149            # Apply refinement options
2150            if G2PwdrData.is_valid_refinement_key(key):
2151                pwdr_set[key] = val
2152            elif G2Phase.is_valid_refinement_key(key):
2153                phase_set[key] = val
2154            elif G2Phase.is_valid_HAP_refinement_key(key):
2155                hap_set[key] = val
2156            else:
2157                raise ValueError("Unknown refinement key", key)
2158
2159        for hist in hists:
2160            hist.set_refinements(pwdr_set)
2161        for phase in phases:
2162            phase.set_refinements(phase_set)
2163        for phase in phases:
2164            phase.set_HAP_refinements(hap_set, hists)
2165
2166        pwdr_clear = {}
2167        phase_clear = {}
2168        hap_clear = {}
2169        for key, val in refinement.get('clear', {}).items():
2170            # Clear refinement options
2171            if G2PwdrData.is_valid_refinement_key(key):
2172                pwdr_clear[key] = val
2173            elif G2Phase.is_valid_refinement_key(key):
2174                phase_clear[key] = val
2175            elif G2Phase.is_valid_HAP_refinement_key(key):
2176                hap_set[key] = val
2177            else:
2178                raise ValueError("Unknown refinement key", key)
2179
2180        for hist in hists:
2181            hist.clear_refinements(pwdr_clear)
2182        for phase in phases:
2183            phase.clear_refinements(phase_clear)
2184        for phase in phases:
2185            phase.clear_HAP_refinements(hap_clear, hists)
2186
2187    def index_ids(self):
2188        self.save()
2189        return G2strIO.GetUsedHistogramsAndPhases(self.filename)
2190
2191    def add_constraint_raw(self, cons_scope, constr):
2192        """Adds a constraint of type consType to the project.
2193        cons_scope should be one of "Hist", "Phase", "HAP", or "Global".
2194
2195        WARNING it does not check the constraint is well-constructed"""
2196        constrs = self.data['Constraints']['data']
2197        if 'Global' not in constrs:
2198            constrs['Global'] = []
2199        constrs[cons_scope].append(constr)
2200
2201    def hold_many(self, vars, type):
2202        """Apply holds for all the variables in vars, for constraint of a given type.
2203
2204        type is passed directly to add_constraint_raw as consType
2205
2206        :param list vars: A list of variables to hold. Either :class:`GSASIIobj.G2VarObj` objects,
2207            string variable specifiers, or arguments for :meth:`make_var_obj`
2208        :param str type: A string constraint type specifier. See
2209            :class:`G2Project.add_constraint_raw`
2210
2211        """
2212        for var in vars:
2213            if isinstance(var, str):
2214                var = self.make_var_obj(var)
2215            elif not isinstance(var, G2obj.G2VarObj):
2216                var = self.make_var_obj(*var)
2217            self.add_constraint_raw(type, [[1.0, var], None, None, 'h'])
2218
2219    def make_var_obj(self, phase=None, hist=None, varname=None, atomId=None,
2220                     reloadIdx=True):
2221        """Wrapper to create a G2VarObj. Takes either a string representation ("p:h:name:a")
2222        or individual names of phase, histogram, varname, and atomId.
2223
2224        Automatically converts string phase, hist, or atom names into the ID required
2225        by G2VarObj."""
2226
2227        if reloadIdx:
2228            self.index_ids()
2229
2230        # If string representation, short circuit
2231        if hist is None and varname is None and atomId is None:
2232            if isinstance(phase, str) and ':' in phase:
2233                return G2obj.G2VarObj(phase)
2234
2235        # Get phase index
2236        phaseObj = None
2237        if isinstance(phase, G2Phase):
2238            phaseObj = phase
2239            phase = G2obj.PhaseRanIdLookup[phase.ranId]
2240        elif phase in self.data['Phases']:
2241            phaseObj = self.phase(phase)
2242            phaseRanId = phaseObj.ranId
2243            phase = G2obj.PhaseRanIdLookup[phaseRanId]
2244
2245        # Get histogram index
2246        if isinstance(hist, G2PwdrData):
2247            hist = G2obj.HistRanIdLookup[hist.ranId]
2248        elif hist in self.data:
2249            histRanId = self.histogram(hist).ranId
2250            hist = G2obj.HistRanIdLookup[histRanId]
2251
2252        # Get atom index (if any)
2253        if isinstance(atomId, G2AtomRecord):
2254            atomId = G2obj.AtomRanIdLookup[phase][atomId.ranId]
2255        elif phaseObj:
2256            atomObj = phaseObj.atom(atomId)
2257            if atomObj:
2258                atomRanId = atomObj.ranId
2259                atomId = G2obj.AtomRanIdLookup[phase][atomRanId]
2260
2261        return G2obj.G2VarObj(phase, hist, varname, atomId)
2262
2263    def add_image(self, imagefile, fmthint=None, defaultImage=None):
2264        """Load an image into a project
2265
2266        :param str imagefile: The image file to read, a filename.
2267        :param str fmthint: If specified, only importers where the format name
2268          (reader.formatName, as shown in Import menu) contains the
2269          supplied string will be tried as importers. If not specified, all
2270          importers consistent with the file extension will be tried
2271          (equivalent to "guess format" in menu).
2272        :param str defaultImage: The name of an image to use as a default for
2273          setting parameters for the image file to read.
2274
2275        :returns: a list of G2Image object for the added image(s) [this routine
2276          has not yet been tested with files with multiple images...]
2277        """
2278        LoadG2fil()
2279        imagefile = os.path.abspath(os.path.expanduser(imagefile))
2280        readers = import_generic(imagefile, Readers['Image'], fmthint=fmthint)
2281        objlist = []
2282        for rd in readers:
2283            if rd.SciPy:        #was default read by scipy; needs 1 time fixes
2284                print('TODO: Image {} read by SciPy. Parameters likely need editing'.format(imagefile))
2285                #see this: G2IO.EditImageParms(self,rd.Data,rd.Comments,rd.Image,imagefile)
2286                rd.SciPy = False
2287            rd.readfilename = imagefile
2288            if rd.repeatcount == 1 and not rd.repeat: # skip image number if only one in set
2289                rd.Data['ImageTag'] = None
2290            else:
2291                rd.Data['ImageTag'] = rd.repeatcount
2292            rd.Data['formatName'] = rd.formatName
2293            if rd.sumfile:
2294                rd.readfilename = rd.sumfile
2295            # Load generic metadata, as configured
2296            G2fil.GetColumnMetadata(rd)
2297            # Code from G2IO.LoadImage2Tree(rd.readfilename,self,rd.Comments,rd.Data,rd.Npix,rd.Image)
2298            Imax = np.amax(rd.Image)
2299            ImgNames = [i[0] for i in self.names if i[0].startswith('IMG ')]
2300            TreeLbl = 'IMG '+os.path.basename(imagefile)
2301            ImageTag = rd.Data.get('ImageTag')
2302            if ImageTag:
2303                TreeLbl += ' #'+'%04d'%(ImageTag)
2304                imageInfo = (imagefile,ImageTag)
2305            else:
2306                imageInfo = imagefile
2307            TreeName = G2obj.MakeUniqueLabel(TreeLbl,ImgNames)
2308            # MT dict to contain image info
2309            ImgDict = {}
2310            ImgDict['data'] = [rd.Npix,imageInfo]
2311            ImgDict['Comments'] = rd.Comments
2312            if defaultImage:
2313                if isinstance(defaultImage, G2Image):
2314                    if defaultImage.proj == self:
2315                        defaultImage = self.data[defaultImage.name]['data']
2316                    else:
2317                        raise Exception("Image {} not in current selected project".format(defaultImage.name))
2318                elif defaultImage in self._images():
2319                    defaultImage = self.data[defaultImage]['data']
2320                else:
2321                    defaultImage = None
2322            Data = rd.Data
2323            if defaultImage:
2324                Data = copy.copy(defaultImage)
2325                Data['showLines'] = True
2326                Data['ring'] = []
2327                Data['rings'] = []
2328                Data['cutoff'] = 10.
2329                Data['pixLimit'] = 20
2330                Data['edgemin'] = 100000000
2331                Data['calibdmin'] = 0.5
2332                Data['calibskip'] = 0
2333                Data['ellipses'] = []
2334                Data['calibrant'] = ''
2335                Data['GonioAngles'] = [0.,0.,0.]
2336                Data['DetDepthRef'] = False
2337            else:
2338                Data['type'] = 'PWDR'
2339                Data['color'] = GSASIIpath.GetConfigValue('Contour_color','Paired')
2340                if 'tilt' not in Data:          #defaults if not preset in e.g. Bruker importer
2341                    Data['tilt'] = 0.0
2342                    Data['rotation'] = 0.0
2343                    Data['pixLimit'] = 20
2344                    Data['calibdmin'] = 0.5
2345                    Data['cutoff'] = 10.
2346                Data['showLines'] = False
2347                Data['calibskip'] = 0
2348                Data['ring'] = []
2349                Data['rings'] = []
2350                Data['edgemin'] = 100000000
2351                Data['ellipses'] = []
2352                Data['GonioAngles'] = [0.,0.,0.]
2353                Data['DetDepth'] = 0.
2354                Data['DetDepthRef'] = False
2355                Data['calibrant'] = ''
2356                Data['IOtth'] = [5.0,50.0]
2357                Data['LRazimuth'] = [0.,180.]
2358                Data['azmthOff'] = 0.0
2359                Data['outChannels'] = 2500
2360                Data['outAzimuths'] = 1
2361                Data['centerAzm'] = False
2362                Data['fullIntegrate'] = GSASIIpath.GetConfigValue('fullIntegrate',True)
2363                Data['setRings'] = False
2364                Data['background image'] = ['',-1.0]                           
2365                Data['dark image'] = ['',-1.0]
2366                Data['Flat Bkg'] = 0.0
2367                Data['Oblique'] = [0.5,False]
2368            Data['setDefault'] = False
2369            Data['range'] = [(0,Imax),[0,Imax]]
2370            ImgDict['Image Controls'] = Data
2371            ImgDict['Masks'] = {'Points':[],'Rings':[],'Arcs':[],'Polygons':[],
2372                                'Frames':[],'Thresholds':[(0,Imax),[0,Imax]]}
2373            ImgDict['Stress/Strain']  = {'Type':'True','d-zero':[],'Sample phi':0.0,
2374                                             'Sample z':0.0,'Sample load':0.0}
2375            self.names.append([TreeName]+['Comments','Image Controls','Masks','Stress/Strain'])
2376            self.data[TreeName] = ImgDict
2377            del rd.Image
2378            objlist.append(G2Image(self.data[TreeName], TreeName, self))
2379        return objlist
2380   
2381    def get_Controls(self, control):
2382        '''Return project controls settings
2383
2384        :param str control: the item to be returned. See below for allowed values.
2385        :returns: The value for the control.
2386
2387        Allowed values for parameter control:
2388
2389            * cycles: the maximum number of cycles (returns int)
2390            * sequential: the histograms used for a sequential refinement as a list
2391              of histogram names or an empty list when in non-sequential mode.
2392            * seqCopy: returns True or False. True indicates that results from
2393              each sequential fit are used as the starting point for the next
2394              histogram.
2395            * Anything else returns the value in the Controls dict, if present. An
2396              exception is raised if the control value is not present.
2397
2398        .. seealso::
2399            :meth:`set_Controls`
2400        '''
2401        if control == 'cycles':
2402            return self.data['Controls']['data']['max cyc']
2403        elif control == 'sequential':
2404            return self.data['Controls']['data']['Seq Data']
2405        elif control in self.data['Controls']['data']:
2406            return self.data['Controls']['data'][control]
2407        elif control == 'seqCopy':
2408            return self.data['Controls']['data']['Copy2Next']
2409        else:
2410            print('Defined Controls:',self.data['Controls']['data'].keys())
2411            raise Exception('{} is an invalid control value'.format(control))
2412       
2413    def set_Controls(self, control, value):
2414        '''Set project controls
2415
2416        :param str control: the item to be set. See below for allowed values.
2417        :param value: the value to be set.
2418
2419        Allowed values for parameter control:
2420
2421            * cycles: sets the maximum number of cycles (value must be int)
2422            * sequential: sets the histograms to be used for a sequential refinement.
2423              Use an empty list to turn off sequential fitting.
2424              The values in the list may be the name of the histogram (a str), or
2425              a ranId or index (int values), see :meth:`histogram`.
2426            * seqCopy: when True, the results from each sequential fit are used as
2427              the starting point for the next. After each fit is is set to False.
2428              Ignored for non-sequential fits.
2429
2430        .. seealso::
2431            :meth:`get_Controls`
2432        '''
2433        if control == 'cycles':
2434            self.data['Controls']['data']['max cyc'] = int(value)
2435        elif control == 'seqCopy':
2436            self.data['Controls']['data']['Copy2Next'] = bool(value)
2437        elif control == 'sequential':
2438            histlist = []
2439            for i,j in enumerate(value):
2440                h = self.histogram(j)
2441                if h:
2442                    histlist.append(h.name)
2443                else:
2444                    raise Exception('item #{} ({}) is an invalid histogram value'
2445                                        .format(i,j))
2446            self.data['Controls']['data']['Seq Data'] = histlist
2447        else:
2448            raise Exception('{} is an invalid control value'.format(control))
2449       
2450    def copyHistParms(self,sourcehist,targethistlist='all',modelist='all'):
2451        '''Copy histogram information from one histogram to others
2452
2453        :param sourcehist: is a histogram object (:class:`G2PwdrData`) or
2454            a histogram name or the index number of the histogram
2455        :param list targethistlist: a list of histograms where each item in the
2456            list can be a histogram object (:class:`G2PwdrData`),
2457            a histogram name or the index number of the histogram.
2458            if the string 'all' (default value), then all histograms in
2459            the project are used.
2460        :param list modelist: May be a list of sections to copy, which may
2461           include 'Background', 'Instrument Parameters', 'Limits' and
2462           'Sample Parameters' (items may be shortened to uniqueness and
2463           capitalization is ignored, so ['b','i','L','s'] will work.)
2464           The default value, 'all' causes the listed sections to
2465
2466        '''
2467        sections = ('Background','Instrument Parameters','Limits',
2468                        'Sample Parameters')
2469        hist_in = self.histogram(sourcehist)
2470        if not hist_in:
2471            raise Exception('{} is not a valid histogram'.format(sourcehist))
2472        if targethistlist == "all":
2473            targethistlist = self.histograms()
2474        if 'all' in modelist:
2475            copysections = sections
2476        else:
2477            copysections = set()
2478            for s in sections:
2479                for m in modelist:
2480                    if s.lower().startswith(m.lower()):
2481                        copysections.add(s)
2482        for h in targethistlist:
2483            hist_out = self.histogram(h)
2484            if not hist_out:
2485                raise Exception('{} is not a valid histogram'.format(h))
2486            for key in copysections: 
2487                hist_out[key] = copy.deepcopy(hist_in[key])
2488       
2489class G2AtomRecord(G2ObjectWrapper):
2490    """Wrapper for an atom record. Has convenient accessors via @property.
2491
2492
2493    Available accessors: label, type, refinement_flags, coordinates,
2494        occupancy, ranId, id, adp_flag, uiso
2495
2496    Example:
2497
2498    >>> atom = some_phase.atom("O3")
2499    >>> # We can access the underlying data format
2500    >>> atom.data
2501    ['O3', 'O-2', '', ... ]
2502    >>> # We can also use wrapper accessors
2503    >>> atom.coordinates
2504    (0.33, 0.15, 0.5)
2505    >>> atom.refinement_flags
2506    u'FX'
2507    >>> atom.ranId
2508    4615973324315876477
2509    >>> atom.occupancy
2510    1.0
2511    """
2512    def __init__(self, data, indices, proj):
2513        self.data = data
2514        self.cx, self.ct, self.cs, self.cia = indices
2515        self.proj = proj
2516
2517    @property
2518    def label(self):
2519        '''Get the associated atom's label
2520        '''
2521        return self.data[self.ct-1]
2522
2523    @property
2524    def type(self):
2525        '''Get the associated atom's type
2526        '''
2527        return self.data[self.ct]
2528
2529    @property
2530    def refinement_flags(self):
2531        '''Get or set refinement flags for the associated atom
2532        '''
2533        return self.data[self.ct+1]
2534
2535    @refinement_flags.setter
2536    def refinement_flags(self, other):
2537        # Automatically check it is a valid refinement
2538        for c in other:
2539            if c not in ' FXU':
2540                raise ValueError("Invalid atom refinement: ", other)
2541        self.data[self.ct+1] = other
2542
2543    @property
2544    def coordinates(self):
2545        '''Get the associated atom's coordinates
2546        '''
2547        return tuple(self.data[self.cx:self.cx+3])
2548
2549    @property
2550    def occupancy(self):
2551        '''Get or set the associated atom's occupancy fraction
2552        '''
2553        return self.data[self.cx+3]
2554
2555    @occupancy.setter
2556    def occupancy(self, val):
2557        self.data[self.cx+3] = float(val)
2558
2559    @property
2560    def ranId(self):
2561        '''Get the associated atom's Random Id number
2562        '''
2563        return self.data[self.cia+8]
2564
2565    @property
2566    def adp_flag(self):
2567        '''Get the associated atom's iso/aniso setting, 'I' or 'A'
2568        '''
2569        # Either 'I' or 'A'
2570        return self.data[self.cia]
2571
2572    @property
2573    def uiso(self):
2574        '''Get or set the associated atom's Uiso or Uaniso value(s)
2575        '''
2576        if self.adp_flag == 'I':
2577            return self.data[self.cia+1]
2578        else:
2579            return self.data[self.cia+2:self.cia+8]
2580
2581    @uiso.setter
2582    def uiso(self, value):
2583        if self.adp_flag == 'I':
2584            self.data[self.cia+1] = float(value)
2585        else:
2586            assert len(value) == 6
2587            self.data[self.cia+2:self.cia+8] = [float(v) for v in value]
2588
2589class G2PwdrData(G2ObjectWrapper):
2590    """Wraps a Powder Data Histogram."""
2591    def __init__(self, data, proj, name):
2592        self.data = data
2593        self.proj = proj
2594        self.name = name
2595
2596    @staticmethod
2597    def is_valid_refinement_key(key):
2598        valid_keys = ['Limits', 'Sample Parameters', 'Background',
2599                      'Instrument Parameters']
2600        return key in valid_keys
2601
2602    #@property
2603    #def name(self):
2604    #    return self.data['data'][-1]
2605
2606    @property
2607    def ranId(self):
2608        return self.data['data'][0]['ranId']
2609
2610    @property
2611    def residuals(self):
2612        '''Provides a dictionary with with the R-factors for this histogram.
2613        Includes the weighted and unweighted profile terms (R, Rb, wR, wRb, wRmin)
2614        as well as the Bragg R-values for each phase (ph:H:Rf and ph:H:Rf^2).
2615        '''
2616        data = self.data['data'][0]
2617        return {key: data[key] for key in data
2618                if key in ['R', 'Rb', 'wR', 'wRb', 'wRmin']
2619                   or ':' in key}
2620
2621    @property
2622    def InstrumentParameters(self):
2623        '''Provides a dictionary with with the Instrument Parameters
2624        for this histogram.
2625        '''
2626        return self.data['Instrument Parameters'][0]
2627
2628    @property
2629    def SampleParameters(self):
2630        '''Provides a dictionary with with the Sample Parameters
2631        for this histogram.
2632        '''
2633        return self.data['Sample Parameters']
2634
2635    @property
2636    def Background(self):
2637        '''Provides a list with with the Background parameters
2638        for this histogram.
2639
2640        :returns: list containing a list and dict with background values
2641        '''
2642        return self.data['Background']
2643
2644    def add_back_peak(self,pos,int,sig,gam,refflags=[]):
2645        '''Adds a background peak to the Background parameters
2646       
2647        :param float pos: position of peak, a 2theta or TOF value
2648        :param float int: integrated intensity of background peak, usually large
2649        :param float sig: Gaussian width of background peak, usually large
2650        :param float gam: Lorentzian width of background peak, usually unused (small)
2651        :param list refflags: a list of 1 to 4 boolean refinement flags for
2652            pos,int,sig & gam, respectively (use [0,1] to refine int only).
2653            Defaults to [] which means nothing is refined.
2654        '''
2655        if 'peaksList' not in self.Background[1]:
2656            self.Background[1]['peaksList'] = []
2657        flags = 4*[False]
2658        for i,f in enumerate(refflags):
2659            if i>3: break
2660            flags[i] = bool(f)
2661        bpk = []
2662        for i,j in zip((pos,int,sig,gam),flags):
2663            bpk += [float(i),j]
2664        self.Background[1]['peaksList'].append(bpk)
2665        self.Background[1]['nPeaks'] = len(self.Background[1]['peaksList'])
2666
2667    def del_back_peak(self,peaknum):
2668        '''Removes a background peak from the Background parameters
2669       
2670        :param int peaknum: the number of the peak (starting from 0)
2671        '''
2672        npks = self.Background[1].get('nPeaks',0)
2673        if peaknum >= npks:
2674            raise Exception('peak {} not found in histogram {}'.format(peaknum,self.name))
2675        del self.Background[1]['peaksList'][peaknum]
2676        self.Background[1]['nPeaks'] = len(self.Background[1]['peaksList'])
2677       
2678    def ref_back_peak(self,peaknum,refflags=[]):
2679        '''Sets refinement flag for a background peak
2680       
2681        :param int peaknum: the number of the peak (starting from 0)
2682        :param list refflags: a list of 1 to 4 boolean refinement flags for
2683            pos,int,sig & gam, respectively. If a flag is not specified
2684            it defaults to False (use [0,1] to refine int only).
2685            Defaults to [] which means nothing is refined.
2686        '''
2687        npks = self.Background[1].get('nPeaks',0)
2688        if peaknum >= npks:
2689            raise Exception('peak {} not found in histogram {}'.format(peaknum,self.name))
2690        flags = 4*[False]
2691        for i,f in enumerate(refflags):
2692            if i>3: break
2693            flags[i] = bool(f)
2694        for i,f in enumerate(flags):
2695            self.Background[1]['peaksList'][peaknum][2*i+1] = f
2696                   
2697    @property
2698    def id(self):
2699        self.proj.update_ids()
2700        return self.data['data'][0]['hId']
2701
2702    @id.setter
2703    def id(self, val):
2704        self.data['data'][0]['hId'] = val
2705
2706    def fit_fixed_points(self):
2707        """Attempts to apply a background fit to the fixed points currently specified."""
2708        def SetInstParms(Inst):
2709            dataType = Inst['Type'][0]
2710            insVary = []
2711            insNames = []
2712            insVals = []
2713            for parm in Inst:
2714                insNames.append(parm)
2715                insVals.append(Inst[parm][1])
2716                if parm in ['U','V','W','X','Y','Z','SH/L','I(L2)/I(L1)','alpha',
2717                    'beta-0','beta-1','beta-q','sig-0','sig-1','sig-2','sig-q',] and Inst[parm][2]:
2718                        Inst[parm][2] = False
2719            instDict = dict(zip(insNames, insVals))
2720            instDict['X'] = max(instDict['X'], 0.01)
2721            instDict['Y'] = max(instDict['Y'], 0.01)
2722            if 'SH/L' in instDict:
2723                instDict['SH/L'] = max(instDict['SH/L'], 0.002)
2724            return dataType, instDict, insVary
2725
2726        bgrnd = self.data['Background']
2727
2728        # Need our fixed points in order
2729        bgrnd[1]['FixedPoints'].sort(key=lambda pair: pair[0])
2730        X = [x for x, y in bgrnd[1]['FixedPoints']]
2731        Y = [y for x, y in bgrnd[1]['FixedPoints']]
2732
2733        limits = self.data['Limits'][1]
2734        if X[0] > limits[0]:
2735            X = [limits[0]] + X
2736            Y = [Y[0]] + Y
2737        if X[-1] < limits[1]:
2738            X += [limits[1]]
2739            Y += [Y[-1]]
2740
2741        # Some simple lookups
2742        controls = self.proj['Controls']['data']
2743        inst, inst2 = self.data['Instrument Parameters']
2744        pwddata = self.data['data'][1]
2745
2746        # Construct the data for background fitting
2747        xBeg = np.searchsorted(pwddata[0], limits[0])
2748        xFin = np.searchsorted(pwddata[0], limits[1])
2749        xdata = pwddata[0][xBeg:xFin]
2750        ydata = si.interp1d(X,Y)(ma.getdata(xdata))
2751
2752        W = [1]*len(xdata)
2753        Z = [0]*len(xdata)
2754
2755        dataType, insDict, insVary = SetInstParms(inst)
2756        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2757
2758        # Do the fit
2759        data = np.array([xdata, ydata, W, Z, Z, Z])
2760        G2pwd.DoPeakFit('LSQ', [], bgrnd, limits, inst, inst2, data,
2761                        prevVaryList=bakVary, controls=controls)
2762
2763        # Post-fit
2764        parmDict = {}
2765        bakType, bakDict, bakVary = G2pwd.SetBackgroundParms(bgrnd)
2766        parmDict.update(bakDict)
2767        parmDict.update(insDict)
2768        pwddata[3][xBeg:xFin] *= 0
2769        pwddata[5][xBeg:xFin] *= 0
2770        pwddata[4][xBeg:xFin] = G2pwd.getBackground('', parmDict, bakType, dataType, xdata)[0]
2771
2772        # TODO adjust pwddata? GSASIIpwdGUI.py:1041
2773        # TODO update background
2774        self.proj.save()
2775
2776    def getdata(self,datatype):
2777        '''Provides access to the histogram data of the selected data type
2778
2779        :param str datatype: must be one of the following values (case is ignored)
2780       
2781           * 'X': the 2theta or TOF values for the pattern
2782           * 'Yobs': the observed intensity values
2783           * 'Yweight': the weights for each data point (1/sigma**2)
2784           * 'Ycalc': the computed intensity values
2785           * 'Background': the computed background values
2786           * 'Residual': the difference between Yobs and Ycalc (obs-calc)
2787
2788        :returns: an numpy MaskedArray with data values of the requested type
2789       
2790        '''
2791        enums = ['x', 'yobs', 'yweight', 'ycalc', 'background', 'residual']
2792        if datatype.lower() not in enums:
2793            raise G2ScriptException("Invalid datatype = "+datatype+" must be one of "+str(enums))
2794        return self.data['data'][1][enums.index(datatype.lower())]
2795       
2796    def y_calc(self):
2797        return self.data['data'][1][3]
2798
2799    def Export(self,fileroot,extension):
2800        '''Write the histogram into a file. The path is specified by fileroot and
2801        extension.
2802       
2803        :param str fileroot: name of the file, optionally with a path (extension is
2804           ignored)
2805        :param str extension: includes '.', must match an extension in global
2806           exportersByExtension['powder'] or a Exception is raised.
2807        :returns: name of file that was written
2808        '''
2809        if extension not in exportersByExtension.get('powder',[]):
2810            raise G2ScriptException('No Writer for file type = "'+extension+'"')
2811        fil = os.path.abspath(os.path.splitext(fileroot)[0]+extension)
2812        obj = exportersByExtension['powder'][extension]
2813        obj.SetFromArray(hist=self.data,histname=self.name)
2814        obj.Writer(self.name,fil)
2815        return fil
2816   
2817    def plot(self, Yobs=True, Ycalc=True, Background=True, Residual=True):
2818        try:
2819            import matplotlib.pyplot as plt
2820        except ImportError:
2821            print('Unable to import matplotlib, skipping plot')
2822            return
2823        data = self.data['data'][1]
2824        if Yobs:
2825            plt.plot(data[0], data[1], label='Yobs')
2826        if Ycalc:
2827            plt.plot(data[0], data[3], label='Ycalc')
2828        if Background:
2829            plt.plot(data[0], data[4], label='Background')
2830        if Residual:
2831            plt.plot(data[0], data[5], label="Residual")
2832
2833    def get_wR(self):
2834        """returns the overall weighted profile R factor for a histogram
2835       
2836        :returns: a wR value as a percentage or None if not defined
2837        """
2838        return self['data'][0].get('wR')
2839
2840    def set_refinements(self, refs):
2841        """Sets the histogram refinement parameter 'key' to the specification 'value'.
2842
2843        :param dict refs: A dictionary of the parameters to be set. See the
2844                          :ref:`Histogram_parameters_table` table for a description of
2845                          what these dictionaries should be.
2846
2847        :returns: None
2848
2849        """
2850        do_fit_fixed_points = False
2851        for key, value in refs.items():
2852            if key == 'Limits':
2853                old_limits = self.data['Limits'][1]
2854                new_limits = value
2855                if isinstance(new_limits, dict):
2856                    if 'low' in new_limits:
2857                        old_limits[0] = new_limits['low']
2858                    if 'high' in new_limits:
2859                        old_limits[1] = new_limits['high']
2860                else:
2861                    old_limits[0], old_limits[1] = new_limits
2862            elif key == 'Sample Parameters':
2863                sample = self.data['Sample Parameters']
2864                for sparam in value:
2865                    if sparam not in sample:
2866                        raise ValueError("Unknown refinement parameter, "
2867                                         + str(sparam))
2868                    sample[sparam][1] = True
2869            elif key == 'Background':
2870                bkg, peaks = self.data['Background']
2871
2872                # If True or False, just set the refine parameter
2873                if value in (True, False):
2874                    bkg[1] = value
2875                    return
2876
2877                if 'type' in value:
2878                    bkg[0] = value['type']
2879                if 'refine' in value:
2880                    bkg[1] = value['refine']
2881                if 'no. coeffs' in value:
2882                    cur_coeffs = bkg[2]
2883                    n_coeffs = value['no. coeffs']
2884                    if n_coeffs > cur_coeffs:
2885                        for x in range(n_coeffs - cur_coeffs):
2886                            bkg.append(0.0)
2887                    else:
2888                        for _ in range(cur_coeffs - n_coeffs):
2889                            bkg.pop()
2890                    bkg[2] = n_coeffs
2891                if 'coeffs' in value:
2892                    bkg[3:] = value['coeffs']
2893                if 'FixedPoints' in value:
2894                    peaks['FixedPoints'] = [(float(a), float(b))
2895                                            for a, b in value['FixedPoints']]
2896                if value.get('fit fixed points', False):
2897                    do_fit_fixed_points = True
2898                if 'peaks' in value:
2899                    for i,flags in enumerate(value['peaks']):
2900                        self.ref_back_peak(i,flags)
2901
2902            elif key == 'Instrument Parameters':
2903                instrument, secondary = self.data['Instrument Parameters']
2904                for iparam in value:
2905                    try:
2906                        instrument[iparam][2] = True
2907                    except IndexError:
2908                        raise ValueError("Invalid key:", iparam)
2909            else:
2910                raise ValueError("Unknown key:", key)
2911        # Fit fixed points after the fact - ensure they are after fixed points
2912        # are added
2913        if do_fit_fixed_points:
2914            # Background won't be fit if refinement flag not set
2915            orig = self.data['Background'][0][1]
2916            self.data['Background'][0][1] = True
2917            self.fit_fixed_points()
2918            # Restore the previous value
2919            self.data['Background'][0][1] = orig
2920
2921    def clear_refinements(self, refs):
2922        """Clears the refinement parameter 'key' and its associated value.
2923
2924        :param dict refs: A dictionary of parameters to clear.
2925          See the :ref:`Histogram_parameters_table` table for what can be specified.
2926        """
2927        for key, value in refs.items():
2928            if key == 'Limits':
2929                old_limits, cur_limits = self.data['Limits']
2930                cur_limits[0], cur_limits[1] = old_limits
2931            elif key == 'Sample Parameters':
2932                sample = self.data['Sample Parameters']
2933                for sparam in value:
2934                    sample[sparam][1] = False
2935            elif key == 'Background':
2936                bkg, peaks = self.data['Background']
2937
2938                # If True or False, just set the refine parameter
2939                if value in (True, False):
2940                    bkg[1] = False
2941                    return
2942
2943                bkg[1] = False
2944                if 'FixedPoints' in value:
2945                    if 'FixedPoints' in peaks:
2946                        del peaks['FixedPoints']
2947                if 'peaks' in value:
2948                    for i in range(len(self.Background[1]['peaksList'])):
2949                        self.ref_back_peak(i,[])
2950            elif key == 'Instrument Parameters':
2951                instrument, secondary = self.data['Instrument Parameters']
2952                for iparam in value:
2953                    instrument[iparam][2] = False
2954            else:
2955                raise ValueError("Unknown key:", key)
2956
2957    def add_peak(self,area,dspace=None,Q=None,ttheta=None):
2958        '''Adds a single peak to the peak list
2959        :param float area: peak area
2960        :param float dspace: peak position as d-space (A)
2961        :param float Q: peak position as Q (A-1)
2962        :param float ttheta: peak position as 2Theta (deg)
2963
2964        Note: only one of the parameters dspace, Q or ttheta may be specified
2965        '''
2966        import GSASIIlattice as G2lat
2967        import GSASIImath as G2mth
2968        if (not dspace) + (not Q) + (not ttheta) != 2:
2969            print('add_peak error: too many or no peak position(s) specified')
2970            return
2971        pos = ttheta
2972        Parms,Parms2 = self.data['Instrument Parameters']
2973        if Q:
2974            pos = G2lat.Dsp2pos(Parms,2.*np.pi/Q)
2975        elif dspace:
2976            pos = G2lat.Dsp2pos(Parms,dspace)
2977        peaks = self.data['Peak List']
2978        peaks['sigDict'] = {}        #no longer valid
2979        peaks['peaks'].append(G2mth.setPeakparms(Parms,Parms2,pos,area))
2980
2981    def set_peakFlags(self,peaklist=None,area=None,pos=None,sig=None,gam=None):
2982        '''Set refinement flags for peaks
2983       
2984        :param list peaklist: a list of peaks to change flags. If None (default), changes
2985          are made to all peaks.
2986        :param bool area: Sets or clears the refinement flag for the peak area value.
2987          If None (the default), no change is made.
2988        :param bool pos: Sets or clears the refinement flag for the peak position value.
2989          If None (the default), no change is made.
2990        :param bool sig: Sets or clears the refinement flag for the peak sig (Gaussian width) value.
2991          If None (the default), no change is made.
2992        :param bool gam: Sets or clears the refinement flag for the peak sig (Lorentzian width) value.
2993          If None (the default), no change is made.
2994         
2995        Note that when peaks are first created the area flag is on and the other flags are
2996        initially off.
2997
2998        Example::
2999       
3000           set_peakFlags(sig=False,gam=True)
3001
3002        causes the sig refinement flag to be cleared and the gam flag to be set, in both cases for
3003        all peaks. The position and area flags are not changed from their previous values.
3004        '''
3005        peaks = self.data['Peak List']
3006        if peaklist is None:
3007            peaklist = range(len(peaks['peaks']))
3008        for i in peaklist:
3009            for var,j in [(area,3),(pos,1),(sig,5),(gam,7)]:
3010                if var is not None:
3011                    peaks['peaks'][i][j] = var
3012           
3013    def refine_peaks(self):
3014        '''Causes a refinement of peak position, background and instrument parameters
3015        '''
3016        import GSASIIpwd as G2pwd
3017        controls = self.proj.data.get('Controls',{})
3018        controls = controls.get('data',
3019                            {'deriv type':'analytic','min dM/M':0.001,}     #fill in defaults if needed
3020                            )
3021        peaks = self.data['Peak List']
3022        Parms,Parms2 = self.data['Instrument Parameters']
3023        background = self.data['Background']
3024        limits = self.data['Limits'][1]
3025        bxye = np.zeros(len(self.data['data'][1][1]))
3026        peaks['sigDict'] = G2pwd.DoPeakFit('LSQ',peaks['peaks'],background,limits,
3027                                           Parms,Parms2,self.data['data'][1],bxye,[],
3028                                           False,controls,None)[0]
3029
3030    def SaveProfile(self,filename):
3031        '''Writes a GSAS-II (new style) .instprm file
3032        '''
3033        data,Parms2 = self.data['Instrument Parameters']
3034        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
3035        File = open(filename,'w')
3036        File.write("#GSAS-II instrument parameter file; do not add/delete items!\n")
3037        for item in data:
3038            File.write(item+':'+str(data[item][1])+'\n')
3039        File.close()
3040        print ('Instrument parameters saved to: '+filename)
3041
3042    def LoadProfile(self,filename,bank=0):
3043        '''Reads a GSAS-II (new style) .instprm file and overwrites the current
3044        parameters
3045
3046        :param str filename: instrument parameter file name, extension ignored if not
3047          .instprm
3048        :param int bank: bank number to read, defaults to zero
3049        '''
3050        filename = os.path.splitext(filename)[0]+'.instprm'         # make sure extension is .instprm
3051        File = open(filename,'r')
3052        S = File.readline()
3053        newItems = []
3054        newVals = []
3055        Found = False
3056        while S:
3057            if S[0] == '#':
3058                if Found:
3059                    break
3060                if 'Bank' in S:
3061                    if bank == int(S.split(':')[0].split()[1]):
3062                        S = File.readline()
3063                        continue
3064                    else:
3065                        S = File.readline()
3066                        while S and '#Bank' not in S:
3067                            S = File.readline()
3068                        continue
3069                else:   #a non #Bank file
3070                    S = File.readline()
3071                    continue
3072            Found = True
3073            [item,val] = S[:-1].split(':')
3074            newItems.append(item)
3075            try:
3076                newVals.append(float(val))
3077            except ValueError:
3078                newVals.append(val)                       
3079            S = File.readline()               
3080        File.close()
3081        LoadG2fil()
3082        self.data['Instrument Parameters'][0] = G2fil.makeInstDict(newItems,newVals,len(newVals)*[False,])
3083
3084       
3085class G2Phase(G2ObjectWrapper):
3086    """A wrapper object around a given phase.
3087
3088    Author: Jackson O'Donnell (jacksonhodonnell .at. gmail.com)
3089    """
3090    def __init__(self, data, name, proj):
3091        self.data = data
3092        self.name = name
3093        self.proj = proj
3094
3095    @staticmethod
3096    def is_valid_refinement_key(key):
3097        valid_keys = ["Cell", "Atoms", "LeBail"]
3098        return key in valid_keys
3099
3100    @staticmethod
3101    def is_valid_HAP_refinement_key(key):
3102        valid_keys = ["Babinet", "Extinction", "HStrain", "Mustrain",
3103                      "Pref.Ori.", "Show", "Size", "Use", "Scale"]
3104        return key in valid_keys
3105
3106    def atom(self, atomlabel):
3107        """Returns the atom specified by atomlabel, or None if it does not
3108        exist.
3109
3110        :param str atomlabel: The name of the atom (e.g. "O2")
3111        :returns: A :class:`G2AtomRecord` object
3112            representing the atom.
3113        """
3114        # Consult GSASIIobj.py for the meaning of this
3115        cx, ct, cs, cia = self.data['General']['AtomPtrs']
3116        ptrs = [cx, ct, cs, cia]
3117        atoms = self.data['Atoms']
3118        for atom in atoms:
3119            if atom[ct-1] == atomlabel:
3120                return G2AtomRecord(atom, ptrs, self.proj)
3121
3122    def atoms(self):
3123        """Returns a list of atoms present in the current phase.
3124
3125        :returns: A list of :class:`G2AtomRecord` objects.
3126
3127        .. seealso::
3128            :meth:`~G2Phase.atom`
3129            :class:`G2AtomRecord`
3130        """
3131        ptrs = self.data['General']['AtomPtrs']
3132        output = []
3133        atoms = self.data['Atoms']
3134        for atom in atoms:
3135            output.append(G2AtomRecord(atom, ptrs, self.proj))
3136        return output
3137
3138    def histograms(self):
3139        '''Returns a list of histogram names associated with the current phase.
3140        '''
3141        output = []
3142        for hname in self.data.get('Histograms', {}).keys():
3143            output.append(self.proj.histogram(hname))
3144        return output
3145
3146    @property
3147    def ranId(self):
3148        return self.data['ranId']
3149
3150    @property
3151    def id(self):
3152        return self.data['pId']
3153
3154    @id.setter
3155    def id(self, val):
3156        self.data['pId'] = val
3157
3158    def get_cell(self):
3159        """Returns a dictionary of the cell parameters, with keys:
3160            'length_a', 'length_b', 'length_c', 'angle_alpha', 'angle_beta', 'angle_gamma', 'volume'
3161
3162        :returns: a dict
3163
3164        .. seealso::
3165           :meth:`~G2Phase.get_cell_and_esd`
3166
3167        """
3168        cell = self.data['General']['Cell']
3169        return {'length_a': cell[1], 'length_b': cell[2], 'length_c': cell[3],
3170                'angle_alpha': cell[4], 'angle_beta': cell[5], 'angle_gamma': cell[6],
3171                'volume': cell[7]}
3172
3173    def get_cell_and_esd(self):
3174        """
3175        Returns a pair of dictionaries, the first representing the unit cell, the second
3176        representing the estimated standard deviations of the unit cell.
3177
3178        :returns: a tuple of two dictionaries
3179
3180        .. seealso::
3181           :meth:`~G2Phase.get_cell`
3182
3183        """
3184        # translated from GSASIIstrIO.ExportBaseclass.GetCell
3185        import GSASIIlattice as G2lat
3186        import GSASIImapvars as G2mv
3187        try:
3188            pfx = str(self.id) + '::'
3189            sgdata = self['General']['SGData']
3190            covDict = self.proj['Covariance']['data']
3191
3192            parmDict = dict(zip(covDict.get('varyList',[]),
3193                                covDict.get('variables',[])))
3194            sigDict = dict(zip(covDict.get('varyList',[]),
3195                               covDict.get('sig',[])))
3196
3197            if covDict.get('covMatrix') is not None:
3198                sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
3199                                                  covDict['varyList'],
3200                                                  parmDict))
3201
3202            A, sigA = G2strIO.cellFill(pfx, sgdata, parmDict, sigDict)
3203            cellSig = G2strIO.getCellEsd(pfx, sgdata, A, self.proj['Covariance']['data'])
3204            cellList = G2lat.A2cell(A) + (G2lat.calc_V(A),)
3205            cellDict, cellSigDict = {}, {}
3206            for i, key in enumerate(['length_a', 'length_b', 'length_c',
3207                                     'angle_alpha', 'angle_beta', 'angle_gamma',
3208                                     'volume']):
3209                cellDict[key] = cellList[i]
3210                cellSigDict[key] = cellSig[i]
3211            return cellDict, cellSigDict
3212        except KeyError:
3213            cell = self.get_cell()
3214            return cell, {key: 0.0 for key in cell}
3215
3216    def export_CIF(self, outputname, quickmode=True):
3217        """Write this phase to a .cif file named outputname
3218
3219        :param str outputname: The name of the .cif file to write to
3220        :param bool quickmode: Currently ignored. Carryover from exports.G2export_CIF"""
3221        # This code is all taken from exports/G2export_CIF.py
3222        # Functions copied have the same names
3223        import GSASIImath as G2mth
3224        import GSASIImapvars as G2mv
3225        from exports import G2export_CIF as cif
3226
3227        CIFdate = dt.datetime.strftime(dt.datetime.now(),"%Y-%m-%dT%H:%M")
3228        CIFname = os.path.splitext(self.proj.filename)[0]
3229        CIFname = os.path.split(CIFname)[1]
3230        CIFname = ''.join([c if ord(c) < 128 else ''
3231                           for c in CIFname.replace(' ', '_')])
3232        try:
3233            author = self.proj['Controls']['data'].get('Author','').strip()
3234        except KeyError:
3235            pass
3236        oneblock = True
3237
3238        covDict = self.proj['Covariance']['data']
3239        parmDict = dict(zip(covDict.get('varyList',[]),
3240                            covDict.get('variables',[])))
3241        sigDict = dict(zip(covDict.get('varyList',[]),
3242                           covDict.get('sig',[])))
3243
3244        if covDict.get('covMatrix') is not None:
3245            sigDict.update(G2mv.ComputeDepESD(covDict['covMatrix'],
3246                                              covDict['varyList'],
3247                                              parmDict))
3248
3249        with open(outputname, 'w') as fp:
3250            fp.write(' \n' + 70*'#' + '\n')
3251            cif.WriteCIFitem(fp, 'data_' + CIFname)
3252            # from exports.G2export_CIF.WritePhaseInfo
3253            cif.WriteCIFitem(fp, '\n# phase info for '+str(self.name) + ' follows')
3254            cif.WriteCIFitem(fp, '_pd_phase_name', self.name)
3255            # TODO get esds
3256            cellDict = self.get_cell()
3257            defsigL = 3*[-0.00001] + 3*[-0.001] + [-0.01] # significance to use when no sigma
3258            names = ['length_a','length_b','length_c',
3259                     'angle_alpha','angle_beta ','angle_gamma',
3260                     'volume']
3261            for key, val in cellDict.items():
3262                cif.WriteCIFitem(fp, '_cell_' + key, G2mth.ValEsd(val))
3263
3264            cif.WriteCIFitem(fp, '_symmetry_cell_setting',
3265                         self.data['General']['SGData']['SGSys'])
3266
3267            spacegroup = self.data['General']['SGData']['SpGrp'].strip()
3268            # regularize capitalization and remove trailing H/R
3269            spacegroup = spacegroup[0].upper() + spacegroup[1:].lower().rstrip('rh ')
3270            cif.WriteCIFitem(fp, '_symmetry_space_group_name_H-M', spacegroup)
3271
3272            # generate symmetry operations including centering and center of symmetry
3273            SymOpList, offsetList, symOpList, G2oprList, G2opcodes = G2spc.AllOps(
3274                self.data['General']['SGData'])
3275            cif.WriteCIFitem(fp, 'loop_\n    _space_group_symop_id\n    _space_group_symop_operation_xyz')
3276            for i, op in enumerate(SymOpList,start=1):
3277                cif.WriteCIFitem(fp, '   {:3d}  {:}'.format(i,op.lower()))
3278
3279            # TODO skipped histograms, exports/G2export_CIF.py:880
3280
3281            # report atom params
3282            if self.data['General']['Type'] in ['nuclear','macromolecular']:        #this needs macromolecular variant, etc!
3283                cif.WriteAtomsNuclear(fp, self.data, self.name, parmDict, sigDict, [])
3284                # self._WriteAtomsNuclear(fp, parmDict, sigDict)
3285            else:
3286                raise G2ScriptException("no export for "+str(self.data['General']['Type'])+" coordinates implemented")
3287            # report cell contents
3288            cif.WriteComposition(fp, self.data, self.name, parmDict)
3289            if not quickmode and self.data['General']['Type'] == 'nuclear':      # report distances and angles
3290                # WriteDistances(fp,self.name,SymOpList,offsetList,symOpList,G2oprList)
3291                raise NotImplementedError("only quickmode currently supported")
3292            if 'Map' in self.data['General'] and 'minmax' in self.data['General']['Map']:
3293                cif.WriteCIFitem(fp,'\n# Difference density results')
3294                MinMax = self.data['General']['Map']['minmax']
3295                cif.WriteCIFitem(fp,'_refine_diff_density_max',G2mth.ValEsd(MinMax[0],-0.009))
3296                cif.WriteCIFitem(fp,'_refine_diff_density_min',G2mth.ValEsd(MinMax[1],-0.009))
3297
3298
3299    def set_refinements(self, refs):
3300        """Sets the phase refinement parameter 'key' to the specification 'value'
3301
3302        :param dict refs: A dictionary of the parameters to be set. See the
3303                          :ref:`Phase_parameters_table` table for a description of
3304                          this dictionary.
3305
3306        :returns: None"""
3307        for key, value in refs.items():
3308            if key == "Cell":
3309                self.data['General']['Cell'][0] = value
3310
3311            elif key == "Atoms":
3312                for atomlabel, atomrefinement in value.items():
3313                    if atomlabel == 'all':
3314                        for atom in self.atoms():
3315                            atom.refinement_flags = atomrefinement
3316                    else:
3317                        atom = self.atom(atomlabel)
3318                        if atom is None:
3319                            raise ValueError("No such atom: " + atomlabel)
3320                        atom.refinement_flags = atomrefinement
3321
3322            elif key == "LeBail":
3323                hists = self.data['Histograms']
3324                for hname, hoptions in hists.items():
3325                    if 'LeBail' not in hoptions:
3326                        hoptions['newLeBail'] = bool(True)
3327                    hoptions['LeBail'] = bool(value)
3328            else:
3329                raise ValueError("Unknown key:", key)
3330
3331    def clear_refinements(self, refs):
3332        """Clears a given set of parameters.
3333
3334        :param dict refs: The parameters to clear.
3335          See the :ref:`Phase_parameters_table` table for what can be specified.
3336        """
3337        for key, value in refs.items():
3338            if key == "Cell":
3339                self.data['General']['Cell'][0] = False
3340            elif key == "Atoms":
3341                cx, ct, cs, cia = self.data['General']['AtomPtrs']
3342
3343                for atomlabel in value:
3344                    atom = self.atom(atomlabel)
3345                    # Set refinement to none
3346                    atom.refinement_flags = ' '
3347            elif key == "LeBail":
3348                hists = self.data['Histograms']
3349                for hname, hoptions in hists.items():
3350                    if 'LeBail' not in hoptions:
3351                        hoptions['newLeBail'] = True
3352                    hoptions['LeBail'] = False
3353            else:
3354                raise ValueError("Unknown key:", key)
3355
3356    def set_HAP_refinements(self, refs, histograms='all'):
3357        """Sets the given HAP refinement parameters between this phase and
3358        the given histograms.
3359
3360        :param dict refs: A dictionary of the parameters to be set. See
3361                          the :ref:`HAP_parameters_table` table for a description of this
3362                          dictionary.
3363        :param histograms: Either 'all' (default) or a list of the histograms
3364            whose HAP parameters will be set with this phase. Histogram and phase
3365            must already be associated
3366
3367        :returns: None
3368        """
3369        if not self.data['Histograms']:
3370            print("Error likely: Phase {} has no linked histograms".format(self.name))
3371            return
3372           
3373        if histograms == 'all':
3374            histograms = self.data['Histograms'].values()
3375        else:
3376            histograms = [self.data['Histograms'][h.name] for h in histograms
3377                          if h.name in self.data['Histograms']]
3378
3379        if not histograms:
3380            print("Skipping HAP set for phase {}, no selected histograms".format(self.name))
3381            return
3382        for key, val in refs.items():
3383            if key == 'Babinet':
3384                try:
3385                    sets = list(val)
3386                except ValueError:
3387                    sets = ['BabA', 'BabU']
3388                for param in sets:
3389                    if param not in ['BabA', 'BabU']:
3390                        raise ValueError("Not sure what to do with" + param)
3391                    for h in histograms:
3392                        h['Babinet'][param][1] = True
3393            elif key == 'Extinction':
3394                for h in histograms:
3395                    h['Extinction'][1] = bool(val)
3396            elif key == 'HStrain':
3397                if isinstance(val,list) or isinstance(val,tuple):
3398                    for h in histograms:
3399                        if len(h['HStrain'][1]) != len(val):
3400                            raise Exception('Need {} HStrain terms for phase {}'
3401                                .format(len(h['HStrain'][1]),self.name))
3402                        for i,v in enumerate(val):
3403                            h['HStrain'][1][i] = bool(v)
3404                else:
3405                    for h in histograms:
3406                        h['HStrain'][1] = [bool(val) for p in h['HStrain'][1]]
3407            elif key == 'Mustrain':
3408                for h in histograms:
3409                    mustrain = h['Mustrain']
3410                    newType = None
3411                    direction = None
3412                    if isinstance(val, strtypes):
3413                        if val in ['isotropic', 'uniaxial', 'generalized']:
3414                            newType = val
3415                        else:
3416                            raise ValueError("Not a Mustrain type: " + val)
3417                    elif isinstance(val, dict):
3418                        newType = val.get('type', None)
3419                        direction = val.get('direction', None)
3420
3421                    if newType:
3422                        mustrain[0] = newType
3423                        if newType == 'isotropic':
3424                            mustrain[2][0] = True == val.get('refine',False)
3425                            mustrain[5] = [False for p in mustrain[4]]
3426                        elif newType == 'uniaxial':
3427                            if 'refine' in val:
3428                                mustrain[2][0] = False
3429                                types = val['refine']
3430                                if isinstance(types, strtypes):
3431                                    types = [types]
3432                                elif isinstance(types, bool):
3433                                    mustrain[2][1] = types
3434                                    mustrain[2][2] = types
3435                                    types = []
3436                                else:
3437                                    raise ValueError("Not sure what to do with: "
3438                                                     + str(types))
3439                            else:
3440                                types = []
3441
3442                            for unitype in types:
3443                                if unitype == 'equatorial':
3444                                    mustrain[2][0] = True
3445                                elif unitype == 'axial':
3446                                    mustrain[2][1] = True
3447                                else:
3448                                    msg = 'Invalid uniaxial mustrain type'
3449                                    raise ValueError(msg + ': ' + unitype)
3450                        else:  # newtype == 'generalized'
3451                            mustrain[2] = [False for p in mustrain[1]]
3452                            if 'refine' in val:
3453                                mustrain[5] = [True == val['refine']]*len(mustrain[5])
3454
3455                    if direction:
3456                        if len(direction) != 3:
3457                            raise ValueError("Expected hkl, found", direction)
3458                        direction = [int(n) for n in direction]
3459                        mustrain[3] = direction
3460            elif key == 'Size':
3461                newSize = None
3462                if 'value' in val:
3463                    newSize = float(val['value'])
3464                for h in histograms:
3465                    size = h['Size']
3466                    newType = None
3467                    direction = None
3468                    if isinstance(val, strtypes):
3469                        if val in ['isotropic', 'uniaxial', 'ellipsoidal']:
3470                            newType = val
3471                        else:
3472                            raise ValueError("Not a valid Size type: " + val)
3473                    elif isinstance(val, dict):
3474                        newType = val.get('type', None)
3475                        direction = val.get('direction', None)
3476
3477                    if newType:
3478                        size[0] = newType
3479                        refine = True == val.get('refine')
3480                        if newType == 'isotropic' and refine is not None:
3481                            size[2][0] = bool(refine)
3482                            if newSize: size[1][0] = newSize
3483                        elif newType == 'uniaxial' and refine is not None:
3484                            size[2][1] = bool(refine)
3485                            size[2][2] = bool(refine)
3486                            if newSize: size[1][1] = size[1][2] =newSize
3487                        elif newType == 'ellipsoidal' and refine is not None:
3488                            size[5] = [bool(refine) for p in size[5]]
3489                            if newSize: size[4] = [newSize for p in size[4]]
3490
3491                    if direction:
3492                        if len(direction) != 3:
3493                            raise ValueError("Expected hkl, found", direction)
3494                        direction = [int(n) for n in direction]
3495                        size[3] = direction
3496            elif key == 'Pref.Ori.':
3497                for h in histograms:
3498                    h['Pref.Ori.'][2] = bool(val)
3499            elif key == 'Show':
3500                for h in histograms:
3501                    h['Show'] = bool(val)
3502            elif key == 'Use':
3503                for h in histograms:
3504                    h['Use'] = bool(val)
3505            elif key == 'Scale':
3506                for h in histograms:
3507                    h['Scale'][1] = bool(val)
3508            else:
3509                print(u'Unknown HAP key: '+key)
3510
3511    def _decodeHist(self,hist):
3512        '''Convert a histogram reference to a histogram name string
3513        '''
3514        if isinstance(hist, G2PwdrData):
3515            return hist.name
3516        elif hist in self.data['Histograms']:
3517            return hist
3518        elif type(hist) is int:
3519            return self.proj.histograms()[hist].name
3520        else:
3521            raise G2ScriptException("Invalid histogram reference: "+str(hist))
3522       
3523    def getHAPvalues(self, histname):
3524        """Returns a dict with HAP values for the selected histogram
3525
3526        :param histogram: is a histogram object (:class:`G2PwdrData`) or
3527            a histogram name or the index number of the histogram
3528
3529        :returns: HAP value dict
3530        """
3531        return self.data['Histograms'][self._decodeHist(histname)]
3532
3533    def copyHAPvalues(self, sourcehist, targethistlist, skip=[]):
3534        """Returns a dict with HAP values for the selected histogram
3535
3536        :param sourcehist: is a histogram object (:class:`G2PwdrData`) or
3537            a histogram name or the index number of the histogram
3538        :param list targethistlist: a list of histograms where each item in the
3539            list can be a histogram object (:class:`G2PwdrData`),
3540            a histogram name or the index number of the histogram.
3541            if the string 'all', then all histograms in the phase are used.
3542        :param list skip: items in the HAP dict that should not be
3543            copied. The default is an empty list, which causes all items
3544            to be copied. To see a list of items in the dict, use
3545            :meth:`getHAPvalues` or use an invalid item, such as '?'.
3546        """
3547        sourcehist = self._decodeHist(sourcehist)
3548        if targethistlist == 'all':
3549            targethistlist = list(self.data['Histograms'].keys())
3550       
3551        copydict = copy.deepcopy(self.data['Histograms'][sourcehist])
3552        for item in skip:
3553            if item in copydict:
3554                del copydict[item]
3555            else:
3556                print('items in HAP dict are: {}'.format(
3557                    list(self.data['Histograms'][sourcehist])))
3558                raise Exception('HAP dict entry {} invalid'.format(item))
3559        for h in targethistlist:
3560            h = self._decodeHist(h)
3561            self.data['Histograms'][h].update(copydict)
3562           
3563        #GSASIIpath.IPyBreak_base()
3564                   
3565    def clear_HAP_refinements(self, refs, histograms='all'):
3566        """Clears the given HAP refinement parameters between this phase and
3567        the given histograms.
3568
3569        :param dict refs: A dictionary of the parameters to be cleared.
3570            See the the :ref:`HAP_parameters_table` table for what can be specified.
3571        :param histograms: Either 'all' (default) or a list of the histograms
3572            whose HAP parameters will be cleared with this phase. Histogram and
3573            phase must already be associated
3574
3575        :returns: None
3576        """
3577        if histograms == 'all':
3578            histograms = self.data['Histograms'].values()
3579        else:
3580            histograms = [self.data['Histograms'][h.name] for h in histograms
3581                          if h.name in self.data['Histograms']]
3582
3583        for key, val in refs.items():
3584            for h in histograms:
3585                if key == 'Babinet':
3586                    try:
3587                        sets = list(val)
3588                    except ValueError:
3589                        sets = ['BabA', 'BabU']
3590                    for param in sets:
3591                        if param not in ['BabA', 'BabU']:
3592                            raise ValueError("Not sure what to do with" + param)
3593                        for hist in histograms:
3594                            hist['Babinet'][param][1] = False
3595                elif key == 'Extinction':
3596                    for h in histograms:
3597                        h['Extinction'][1] = False
3598                elif key == 'HStrain':
3599                    for h in histograms:
3600                        h['HStrain'][1] = [False for p in h['HStrain'][1]]
3601                elif key == 'Mustrain':
3602                    for h in histograms:
3603                        mustrain = h['Mustrain']
3604                        mustrain[2] = [False for p in mustrain[2]]
3605                        mustrain[5] = [False for p in mustrain[4]]
3606                elif key == 'Pref.Ori.':
3607                    for h in histograms:
3608                        h['Pref.Ori.'][2] = False
3609                elif key == 'Show':
3610                    for h in histograms:
3611                        h['Show'] = False
3612                elif key == 'Size':
3613                    for h in histograms:
3614                        size = h['Size']
3615                        size[2] = [False for p in size[2]]
3616                        size[5] = [False for p in size[5]]
3617                elif key == 'Use':
3618                    for h in histograms:
3619                        h['Use'] = False
3620                elif key == 'Scale':
3621                    for h in histograms:
3622                        h['Scale'][1] = False
3623                else:
3624                    print(u'Unknown HAP key: '+key)
3625
3626class G2Image(G2ObjectWrapper):
3627    """Wrapper for an IMG tree entry, containing an image and various metadata.
3628    Note that in a GSASIIscriptable script, instances of G2Image will be created by
3629    calls to :meth:`G2Project.add_image` or :meth:`G2Project.images`, not via calls
3630    to :meth:`G2Image.__init__`.
3631
3632    Example use of G2Image:
3633
3634    >>> gpx = G2sc.G2Project(filename='itest.gpx')
3635    >>> imlst = gpx.add_image(idata,fmthint="TIF")
3636    >>> imlst[0].loadControls('stdSettings.imctrl')
3637    >>> imlst[0].setCalibrant('Si    SRM640c')
3638    >>> imlst[0].loadMasks('stdMasks.immask')
3639    >>> imlst[0].Recalibrate()
3640    >>> imlst[0].setControl('outAzimuths',3)
3641    >>> pwdrList = imlst[0].Integrate()
3642
3643
3644    Sample script using an image:   
3645
3646.. code-block::  python
3647
3648    import os,sys
3649    sys.path.insert(0,'/Users/toby/software/G2/GSASII')
3650    import GSASIIscriptable as G2sc
3651    datadir = "/tmp/V4343_IntegrationError/"
3652    PathWrap = lambda fil: os.path.join(datadir,fil)
3653
3654    gpx = G2sc.G2Project(filename=PathWrap('inttest.gpx'))
3655    imlst = gpx.add_image(PathWrap('Si_free_dc800_1-00000.tif'),fmthint="TIF")
3656    imlst[0].loadControls(PathWrap('Si_free_dc800_1-00000.imctrl'))
3657    pwdrList = imlst[0].Integrate()
3658    gpx.save()
3659
3660    """
3661    # parameters in that can be accessed via setControl. This may need future attention
3662    ControlList = {
3663        'int': ['calibskip', 'pixLimit', 'edgemin', 'outChannels',
3664                    'outAzimuths'],
3665        'float': ['cutoff', 'setdist', 'wavelength', 'Flat Bkg',
3666                      'azmthOff', 'tilt', 'calibdmin', 'rotation',
3667                      'distance', 'DetDepth'],
3668        'bool': ['setRings', 'setDefault', 'centerAzm', 'fullIntegrate',
3669                     'DetDepthRef', 'showLines'],
3670        'str': ['SampleShape', 'binType', 'formatName', 'color',
3671                    'type', ],
3672        'list': ['GonioAngles', 'IOtth', 'LRazimuth', 'Oblique', 'PolaVal',
3673                   'SampleAbs', 'center', 'ellipses', 'linescan',
3674                    'pixelSize', 'range', 'ring', 'rings', 'size', ],
3675        'dict': ['varyList'],
3676        }
3677    '''Defines the items known to exist in the Image Controls tree section
3678    and the item's data types. A few are not included here
3679    ('background image', 'dark image', 'Gain map', and 'calibrant') because
3680    these items have special set routines,
3681    where references to entries are checked to make sure their values are
3682    correct.
3683    ''' 
3684       
3685    def __init__(self, data, name, proj):
3686        self.data = data
3687        self.name = name
3688        self.proj = proj
3689
3690    def setControl(self,arg,value):
3691        '''Set an Image Controls parameter in the current image.
3692        If the parameter is not found an exception is raised.
3693
3694        :param str arg: the name of a parameter (dict entry) in the
3695          image. The parameter must be found in :data:`ControlList`
3696          or an exception is raised.
3697        :param value: the value to set the parameter. The value is
3698          cast as the appropriate type from :data:`ControlList`.
3699        '''
3700        for typ in self.ControlList:
3701            if arg in self.ControlList[typ]: break
3702        else:
3703            print('Allowed args:\n',[nam for nam,typ in self.findControl('')])
3704            raise Exception('arg {} not defined in G2Image.setControl'
3705                                .format(arg))
3706        try:
3707            if typ == 'int':
3708                self.data['Image Controls'][arg] = int(value)
3709            elif typ == 'float':
3710                self.data['Image Controls'][arg] = float(value)
3711            elif typ == 'bool':
3712                self.data['Image Controls'][arg] = bool(value)
3713            elif typ == 'str':
3714                self.data['Image Controls'][arg] = str(value)
3715            elif typ == 'list':
3716                self.data['Image Controls'][arg] = list(value)
3717            elif typ == 'dict':
3718                self.data['Image Controls'][arg] = dict(value)
3719            else:
3720                raise Exception('Unknown type {} for arg {} in  G2Image.setControl'
3721                                    .format(typ,arg))
3722        except:
3723            raise Exception('Error formatting value {} as type {} for arg {} in  G2Image.setControl'
3724                                    .format(value,typ,arg))
3725
3726    def getControl(self,arg):
3727        '''Return an Image Controls parameter in the current image.
3728        If the parameter is not found an exception is raised.
3729
3730        :param str arg: the name of a parameter (dict entry) in the
3731          image.
3732        :returns: the value as a int, float, list,...
3733        '''
3734        if arg in self.data['Image Controls']:
3735            return self.data['Image Controls'][arg]
3736        print(self.findControl(''))
3737        raise Exception('arg {} not defined in G2Image.getControl'.format(arg))
3738
3739    def findControl(self,arg=''):
3740        '''Finds the Image Controls parameter(s) in the current image
3741        that match the string in arg. Default is '' which returns all
3742        parameters.
3743
3744            Example:
3745
3746            >>> findControl('calib')
3747            [['calibskip', 'int'], ['calibdmin', 'float'], ['calibrant', 'str']]
3748
3749        :param str arg: a string containing part of the name of a
3750          parameter (dict entry) in the image's Image Controls.
3751        :returns: a list of matching entries in form
3752          [['item','type'], ['item','type'],...] where each 'item' string
3753          contains the sting in arg.
3754        '''
3755        matchList = []
3756        for typ in self.ControlList:
3757            for item in self.ControlList[typ]:
3758                if arg in item:
3759                    matchList.append([item,typ])
3760        return matchList
3761
3762    def setCalibrant(self,calib):
3763        '''Set a calibrant for the current image
3764
3765        :param str calib: specifies a calibrant name which must be one of
3766          the entries in file ImageCalibrants.py. This is validated and
3767          an error provides a list of valid choices.
3768        '''
3769        import ImageCalibrants as calFile
3770        if calib in calFile.Calibrants.keys():
3771            self.data['Image Controls']['calibrant'] = calib
3772            return
3773        print('Calibrant {} is not valid. Valid calibrants'.format(calib))
3774        for i in calFile.Calibrants.keys():
3775            if i: print('\t"{}"'.format(i))
3776       
3777    def setControlFile(self,typ,imageRef,mult=None):
3778        '''Set a image to be used as a background/dark/gain map image
3779
3780        :param str typ: specifies image type, which must be one of:
3781           'background image', 'dark image', 'gain map'; N.B. only the first
3782           four characters must be specified and case is ignored.
3783        :param imageRef: A reference to the desired image. Either the Image
3784          tree name (str), the image's index (int) or
3785          a image object (:class:`G2Image`)
3786        :param float mult: a multiplier to be applied to the image (not used
3787          for 'Gain map'; required for 'background image', 'dark image'
3788        '''
3789        if 'back' in typ.lower():
3790            key = 'background image'
3791            mult = float(mult)
3792        elif 'dark' in typ.lower():
3793            key = 'dark image'
3794            mult = float(mult)
3795        elif 'gain' in typ.lower():
3796            #key = 'Gain map'
3797            if mult is not None:
3798                print('Ignoring multiplier for Gain map')
3799            mult = None
3800        else:
3801            raise Exception("Invalid typ {} for setControlFile".format(typ))
3802        imgNam = self.proj.image(imageRef).name
3803        if mult is None:
3804            self.data['Image Controls']['Gain map'] = imgNam
3805        else:
3806            self.data['Image Controls'][key] = [imgNam,mult]
3807
3808    def loadControls(self,filename=None,imgDict=None):
3809        '''load controls from a .imctrl file
3810
3811        :param str filename: specifies a file to be read, which should end
3812          with .imctrl (defaults to None, meaning parameters are input
3813          with imgDict.)
3814        :param dict imgDict: contains a set of image parameters (defaults to
3815          None, meaning parameters are input with filename.)
3816        '''
3817        if filename:
3818            File = open(filename,'r')
3819            Slines = File.readlines()
3820            File.close()
3821            G2fil.LoadControls(Slines,self.data['Image Controls'])
3822            print('file {} read into {}'.format(filename,self.name))
3823        elif imgDict:
3824            self.data['Image Controls'].update(imgDict)
3825            print('Image controls set')
3826        else:
3827            raise Exception("loadControls called without imgDict or filename specified")
3828
3829    def saveControls(self,filename):
3830        '''write current controls values to a .imctrl file
3831
3832        :param str filename: specifies a file to write, which should end
3833          with .imctrl
3834        '''
3835        G2fil.WriteControls(filename,self.data['Image Controls'])
3836        print('file {} written from {}'.format(filename,self.name))
3837
3838    def loadMasks(self,filename,ignoreThreshold=False):
3839        '''load masks from a .immask file
3840
3841        :param str filename: specifies a file to be read, which should end
3842          with .immask
3843        :param bool ignoreThreshold: If True, masks are loaded with
3844          threshold masks. Default is False which means any Thresholds
3845          in the file are ignored.
3846        '''
3847        G2fil.readMasks(filename,self.data['Masks'],ignoreThreshold)
3848        print('file {} read into {}'.format(filename,self.name))
3849       
3850    def getVary(self,*args):
3851        '''Return the refinement flag(s) for Image Controls parameter(s)
3852        in the current image.
3853        If the parameter is not found, an exception is raised.
3854
3855        :param str arg: the name of a refinement parameter in the
3856          varyList for the image. The name should be one of
3857          'dep', 'det-X', 'det-Y', 'dist', 'phi', 'tilt', or 'wave'
3858        :param str arg1: the name of a parameter (dict entry) as before,
3859          optional
3860
3861
3862        :returns: a list of bool value(s)
3863        '''
3864        res = []
3865        for arg in args:
3866            if arg in self.data['Image Controls']['varyList']:
3867                res.append(self.data['Image Controls']['varyList'][arg])
3868            else:
3869                raise Exception('arg {} not defined in G2Image.getVary'.format(arg))
3870        return res
3871   
3872    def setVary(self,arg,value):
3873        '''Set a refinement flag for Image Controls parameter in the
3874        current image.
3875        If the parameter is not found an exception is raised.
3876
3877        :param str arg: the name of a refinement parameter in the
3878          varyList for the image. The name should be one of
3879          'dep', 'det-X', 'det-Y', 'dist', 'phi', 'tilt', or 'wave'
3880        :param str arg: the name of a parameter (dict entry) in the
3881          image. The parameter must be found in :data:`ControlList`
3882          or an exception is raised.
3883        :param value: the value to set the parameter. The value is
3884          cast as the appropriate type from :data:`ControlList`.
3885        '''
3886        if arg in self.data['Image Controls']['varyList']:
3887            self.data['Image Controls']['varyList'][arg] = bool(value)
3888        else:
3889            raise Exception('arg {} not defined in G2Image.setVary'.format(arg))
3890
3891    def Recalibrate(self):
3892        '''Invokes a recalibration fit (same as Image Controls/Calibration/Recalibrate
3893        menu command). Note that for this to work properly, the calibration
3894        coefficients (center, wavelength, ditance & tilts) must be fairly close.
3895        This may produce a better result if run more than once.
3896        '''
3897        LoadG2fil()
3898        ImageZ = _getCorrImage(Readers['Image'],self.proj,self)
3899        G2img.ImageRecalibrate(None,ImageZ,self.data['Image Controls'],self.data['Masks'])
3900
3901    def Integrate(self,name=None):
3902        '''Invokes an image integration (same as Image Controls/Integration/Integrate
3903        menu command). All parameters will have previously been set with Image Controls
3904        so no input is needed here. Note that if integration is performed on an
3905        image more than once, histogram entries may be overwritten. Use the name
3906        parameter to prevent this if desired.
3907
3908        :param str name: base name for created histogram(s). If None (default),
3909          the histogram name is taken from the image name.
3910        :returns: a list of created histogram (:class:`G2PwdrData`) objects.
3911        '''
3912        blkSize = 256   #256 seems to be optimal; will break in polymask if >1024
3913        ImageZ = _getCorrImage(Readers['Image'],self.proj,self)
3914        # do integration
3915        ints,azms,Xvals,cancel = G2img.ImageIntegrate(ImageZ,self.data['Image Controls'],self.data['Masks'],blkSize=blkSize)
3916        # code from here on based on G2IO.SaveIntegration, but places results in the current
3917        # project rather than tree
3918        X = Xvals[:-1]
3919        N = len(X)
3920
3921        data = self.data['Image Controls']
3922        Comments = self.data['Comments']
3923        # make name from image, unless overridden
3924        if name:
3925            if not name.startswith(data['type']+' '):
3926                name = data['type']+' '+name
3927        else:
3928            name = self.name.replace('IMG ',data['type']+' ')
3929        if 'PWDR' in name:
3930            if 'target' in data:
3931                names = ['Type','Lam1','Lam2','I(L2)/I(L1)','Zero','Polariz.','U','V','W','X','Y','Z','SH/L','Azimuth'] 
3932                codes = [0 for i in range(14)]
3933            else:
3934                names = ['Type','Lam','Zero','Polariz.','U','V','W','X','Y','Z','SH/L','Azimuth'] 
3935                codes = [0 for i in range(12)]
3936        elif 'SASD' in name:
3937            names = ['Type','Lam','Zero','Azimuth'] 
3938            codes = [0 for i in range(4)]
3939            X = 4.*np.pi*npsind(X/2.)/data['wavelength']    #convert to q
3940        Xminmax = [X[0],X[-1]]
3941        Azms = []
3942        dazm = 0.
3943        if data['fullIntegrate'] and data['outAzimuths'] == 1:
3944            Azms = [45.0,]                              #a poor man's average?
3945        else:
3946            for i,azm in enumerate(azms[:-1]):
3947                if azm > 360. and azms[i+1] > 360.:
3948                    Azms.append(G2img.meanAzm(azm%360.,azms[i+1]%360.))
3949                else:   
3950                    Azms.append(G2img.meanAzm(azm,azms[i+1]))
3951            dazm = np.min(np.abs(np.diff(azms)))/2.
3952        # pull out integration results and make histograms for each
3953        IntgOutList = []
3954        for i,azm in enumerate(azms[:-1]):
3955            Aname = name+" Azm= %.2f"%((azm+dazm)%360.)
3956            # MT dict to contain histogram
3957            HistDict = {}
3958            histItems = [Aname]
3959            Sample = G2obj.SetDefaultSample()       #set as Debye-Scherrer
3960            Sample['Gonio. radius'] = data['distance']
3961            Sample['Omega'] = data['GonioAngles'][0]
3962            Sample['Chi'] = data['GonioAngles'][1]
3963            Sample['Phi'] = data['GonioAngles'][2]
3964            Sample['Azimuth'] = (azm+dazm)%360.    #put here as bin center
3965            polariz = 0.99    #set default polarization for synchrotron radiation!
3966            for item in Comments:
3967                if 'polariz' in item:
3968                    try:
3969                        polariz = float(item.split('=')[1])
3970                    except:
3971                        polariz = 0.99
3972                for key in ('Temperature','Pressure','Time','FreePrm1','FreePrm2','FreePrm3','Omega',
3973                    'Chi','Phi'):
3974                    if key.lower() in item.lower():
3975                        try:
3976                            Sample[key] = float(item.split('=')[1])
3977                        except:
3978                            pass
3979                # if 'label_prm' in item.lower():
3980                #     for num in ('1','2','3'):
3981                #         if 'label_prm'+num in item.lower():
3982                #             Controls['FreePrm'+num] = item.split('=')[1].strip()
3983            if 'PWDR' in Aname:
3984                if 'target' in data:    #from lab x-ray 2D imaging data
3985                    waves = {'CuKa':[1.54051,1.54433],'TiKa':[2.74841,2.75207],'CrKa':[2.28962,2.29351],
3986                                 'FeKa':[1.93597,1.93991],'CoKa':[1.78892,1.79278],'MoKa':[0.70926,0.713543],
3987                                 'AgKa':[0.559363,0.563775]}
3988                    wave1,wave2 = waves[data['target']]
3989                    parms = ['PXC',wave1,wave2,0.5,0.0,polariz,290.,-40.,30.,6.,-14.,0.0,0.0001,Azms[i]]
3990                else:
3991                    parms = ['PXC',data['wavelength'],0.0,polariz,1.0,-0.10,0.4,0.30,1.0,0.0,0.0001,Azms[i]]
3992            elif 'SASD' in Aname:
3993                Sample['Trans'] = data['SampleAbs'][0]
3994                parms = ['LXC',data['wavelength'],0.0,Azms[i]]
3995            Y = ints[i]
3996            Ymin = np.min(Y)
3997            Ymax = np.max(Y)
3998            W = np.where(Y>0.,1./Y,1.e-6)                    #probably not true
3999            section = 'Comments'
4000            histItems += [section]
4001            HistDict[section] = Comments
4002            section = 'Limits'
4003            histItems += [section]
4004            HistDict[section] = copy.deepcopy([tuple(Xminmax),Xminmax])
4005            if 'PWDR' in Aname:
4006                section = 'Background'
4007                histItems += [section]
4008                HistDict[section] = [['chebyschev',1,3,1.0,0.0,0.0],
4009                    {'nDebye':0,'debyeTerms':[],'nPeaks':0,'peaksList':[]}]
4010            inst = [dict(zip(names,zip(parms,parms,codes))),{}]
4011            for item in inst[0]:
4012                inst[0][item] = list(inst[0][item])
4013            section = 'Instrument Parameters'
4014            histItems += [section]
4015            HistDict[section] = inst
4016            if 'PWDR' in Aname:
4017                section = 'Sample Parameters'
4018                histItems += [section]
4019                HistDict[section] = Sample
4020                section = 'Peak List'
4021                histItems += [section]
4022                HistDict[section] = {'sigDict':{},'peaks':[]}
4023                section = 'Index Peak List'
4024                histItems += [section]
4025                HistDict[section] = [[],[]]
4026                section = 'Unit Cells List'
4027                histItems += [section]
4028                HistDict[section] = []
4029                section = 'Reflection Lists'
4030                histItems += [section]
4031                HistDict[section] = {}
4032            # elif 'SASD' in Aname:             
4033            #     section = 'Substances'
4034            #     histItems += [section]
4035            #     HistDict[section] = G2pdG.SetDefaultSubstances()  # this needs to be moved
4036            #     section = 'Sample Parameters'
4037            #     histItems += [section]
4038            #     HistDict[section] = Sample
4039            #     section = 'Models'
4040            #     histItems += [section]
4041            #     HistDict[section] = G2pdG.SetDefaultSASDModel() # this needs to be moved
4042            valuesdict = {
4043                'wtFactor':1.0,'Dummy':False,'ranId':ran.randint(0,sys.maxsize),'Offset':[0.0,0.0],'delOffset':0.02*Ymax,
4044                'refOffset':-0.1*Ymax,'refDelt':0.1*Ymax,'Yminmax':[Ymin,Ymax]}
4045            # if Aname is already in the project replace it
4046            for j in self.proj.names:
4047                if j[0] == Aname: 
4048                    print('Replacing "{}" in project'.format(Aname))
4049                    break
4050            else:
4051                print('Adding "{}" to project'.format(Aname))
4052                self.proj.names.append([Aname]+
4053                        [u'Comments',u'Limits',u'Background',u'Instrument Parameters',
4054                         u'Sample Parameters', u'Peak List', u'Index Peak List',
4055                         u'Unit Cells List', u'Reflection Lists'])
4056            HistDict['data'] = [valuesdict,
4057                    [np.array(X),np.array(Y),np.array(W),np.zeros(N),np.zeros(N),np.zeros(N)]]
4058            self.proj.data[Aname] = HistDict
4059            IntgOutList.append(self.proj.histogram(Aname))
4060        return IntgOutList
4061
4062##########################
4063# Command Line Interface #
4064##########################
4065# Each of these takes an argparse.Namespace object as their argument,
4066# representing the parsed command-line arguments for the relevant subcommand.
4067# The argument specification for each is in the subcommands dictionary (see
4068# below)
4069
4070commandhelp={}
4071commandhelp["create"] = "creates a GSAS-II project, optionally adding histograms and/or phases"
4072def create(args):
4073    """Implements the create command-line subcommand. This creates a GSAS-II project, optionally adding histograms and/or phases::
4074
4075  usage: GSASIIscriptable.py create [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
4076                                  [-i IPARAMS [IPARAMS ...]]
4077                                  [-p PHASES [PHASES ...]]
4078                                  filename
4079                                 
4080positional arguments::
4081
4082  filename              the project file to create. should end in .gpx
4083
4084optional arguments::
4085
4086  -h, --help            show this help message and exit
4087  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
4088                        list of datafiles to add as histograms
4089  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
4090                        instrument parameter file, must be one for every
4091                        histogram
4092  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
4093                        list of phases to add. phases are automatically
4094                        associated with all histograms given.
4095
4096    """
4097    proj = G2Project(gpxfile=args.filename)
4098
4099    hist_objs = []
4100    if args.histograms:
4101        for h,i in zip(args.histograms,args.iparams):
4102            print("Adding histogram from",h,"with instparm ",i)
4103            hist_objs.append(proj.add_powder_histogram(h, i))
4104
4105    if args.phases: 
4106        for p in args.phases:
4107            print("Adding phase from",p)
4108            proj.add_phase(p, histograms=hist_objs)
4109        print('Linking phase(s) to histogram(s):')
4110        for h in hist_objs:
4111            print ('   '+h.name)
4112
4113    proj.save()
4114
4115commandhelp["add"] = "adds histograms and/or phases to GSAS-II project"
4116def add(args):
4117    """Implements the add command-line subcommand. This adds histograms and/or phases to GSAS-II project::
4118
4119  usage: GSASIIscriptable.py add [-h] [-d HISTOGRAMS [HISTOGRAMS ...]]
4120                               [-i IPARAMS [IPARAMS ...]]
4121                               [-hf HISTOGRAMFORMAT] [-p PHASES [PHASES ...]]
4122                               [-pf PHASEFORMAT] [-l HISTLIST [HISTLIST ...]]
4123                               filename
4124
4125
4126positional arguments::
4127
4128  filename              the project file to open. Should end in .gpx
4129
4130optional arguments::
4131
4132  -h, --help            show this help message and exit
4133  -d HISTOGRAMS [HISTOGRAMS ...], --histograms HISTOGRAMS [HISTOGRAMS ...]
4134                        list of datafiles to add as histograms
4135  -i IPARAMS [IPARAMS ...], --iparams IPARAMS [IPARAMS ...]
4136                        instrument parameter file, must be one for every
4137                        histogram
4138  -hf HISTOGRAMFORMAT, --histogramformat HISTOGRAMFORMAT
4139                        format hint for histogram import. Applies to all
4140                        histograms
4141  -p PHASES [PHASES ...], --phases PHASES [PHASES ...]
4142                        list of phases to add. phases are automatically
4143                        associated with all histograms given.
4144  -pf PHASEFORMAT, --phaseformat PHASEFORMAT
4145                        format hint for phase import. Applies to all phases.
4146                        Example: -pf CIF
4147  -l HISTLIST [HISTLIST ...], --histlist HISTLIST [HISTLIST ...]
4148                        list of histgram indices to associate with added
4149                        phases. If not specified, phases are associated with
4150                        all previously loaded histograms. Example: -l 2 3 4
4151   
4152    """
4153    proj = G2Project(args.filename)
4154
4155    if args.histograms:
4156        for h,i in zip(args.histograms,args.iparams):
4157            print("Adding histogram from",h,"with instparm ",i)
4158            proj.add_powder_histogram(h, i, fmthint=args.histogramformat)
4159
4160    if args.phases: 
4161        if not args.histlist:
4162            histlist = proj.histograms()
4163        else:
4164            histlist = [proj.histogram(i) for i in args.histlist]
4165
4166        for p in args.phases:
4167            print("Adding phase from",p)
4168            proj.add_phase(p, histograms=histlist, fmthint=args.phaseformat)
4169           
4170        if not args.histlist:
4171            print('Linking phase(s) to all histogram(s)')
4172        else:
4173            print('Linking phase(s) to histogram(s):')
4174            for h in histlist:
4175                print ('   '+h.name)
4176
4177    proj.save()
4178
4179
4180commandhelp["dump"] = "Shows the contents of a GSAS-II project"
4181def dump(args):
4182    """Implements the dump command-line subcommand, which shows the contents of a GSAS-II project::
4183
4184       usage: GSASIIscriptable.py dump [-h] [-d] [-p] [-r] files [files ...]
4185
4186positional arguments::
4187
4188  files
4189
4190optional arguments::
4191
4192  -h, --help        show this help message and exit
4193  -d, --histograms  list histograms in files, overrides --raw
4194  -p, --phases      list phases in files, overrides --raw
4195  -r, --raw         dump raw file contents, default
4196 
4197    """
4198    if not args.histograms and not args.phases:
4199        args.raw = True
4200    if args.raw:
4201        import IPython.lib.pretty as pretty
4202
4203    for fname in args.files:
4204        if args.raw:
4205            proj, nameList = LoadDictFromProjFile(fname)
4206            print("file:", fname)
4207            print("names:", nameList)
4208            for key, val in proj.items():
4209                print(key, ":")
4210                pretty.pprint(val)
4211        else:
4212            proj = G2Project(fname)
4213            if args.histograms:
4214                hists = proj.histograms()
4215                for h in hists:
4216                    print(fname, "hist", h.id, h.name)
4217            if args.phases:
4218                phase_list = proj.phases()
4219                for p in phase_list:
4220                    print(fname, "phase", p.id, p.name)
4221
4222
4223commandhelp["browse"] = "Load a GSAS-II project and then open a IPython shell to browse it"
4224def IPyBrowse(args):
4225    """Load a .gpx file and then open a IPython shell to browse it::
4226
4227  usage: GSASIIscriptable.py browse [-h] files [files ...]
4228
4229positional arguments::
4230
4231  files       list of files to browse
4232
4233optional arguments::
4234
4235  -h, --help  show this help message and exit
4236
4237    """
4238    for fname in args.files:
4239        proj, nameList = LoadDictFromProjFile(fname)
4240        msg = "\nfname {} loaded into proj (dict) with names in nameList".format(fname)
4241        GSASIIpath.IPyBreak_base(msg)
4242        break
4243
4244
4245commandhelp["refine"] = '''
4246Conducts refinements on GSAS-II projects according to a list of refinement
4247steps in a JSON dict
4248'''
4249def refine(args):
4250    """Implements the refine command-line subcommand:
4251    conducts refinements on GSAS-II projects according to a JSON refinement dict::
4252
4253        usage: GSASIIscriptable.py refine [-h] gpxfile [refinements]
4254
4255positional arguments::
4256
4257  gpxfile      the project file to refine
4258  refinements  json file of refinements to apply. if not present refines file
4259               as-is
4260
4261optional arguments::
4262
4263  -h, --help   show this help message and exit
4264 
4265    """
4266    proj = G2Project(args.gpxfile)
4267    if args.refinements is None:
4268        proj.refine()
4269    else:
4270        import json
4271        with open(args.refinements) as refs:
4272            refs = json.load(refs)
4273        if type(refs) is not dict:
4274            raise G2ScriptException("Error: JSON object must be a dict.")
4275        if "code" in refs:
4276            print("executing code:\n|  ",'\n|  '.join(refs['code']))
4277            exec('\n'.join(refs['code']))
4278        proj.do_refinements(refs['refinements'])
4279
4280
4281commandhelp["export"] = "Export phase as CIF"
4282def export(args):
4283    """Implements the export command-line subcommand: Exports phase as CIF::
4284
4285      usage: GSASIIscriptable.py export [-h] gpxfile phase exportfile
4286
4287positional arguments::
4288
4289  gpxfile     the project file from which to export
4290  phase       identifier of phase to export
4291  exportfile  the .cif file to export to
4292
4293optional arguments::
4294
4295  -h, --help  show this help message and exit
4296
4297    """
4298    proj = G2Project(args.gpxfile)
4299    phase = proj.phase(args.phase)
4300    phase.export_CIF(args.exportfile)
4301
4302
4303def _args_kwargs(*args, **kwargs):
4304    return args, kwargs
4305
4306# A dictionary of the name of each subcommand, and a tuple
4307# of its associated function and a list of its arguments
4308# The arguments are passed directly to the add_argument() method
4309# of an argparse.ArgumentParser
4310
4311subcommands = {"create":
4312               (create, [_args_kwargs('filename',
4313                                      help='the project file to create. should end in .gpx'),
4314
4315                         _args_kwargs('-d', '--histograms',
4316                                      nargs='+',
4317                                      help='list of datafiles to add as histograms'),
4318                                     
4319                         _args_kwargs('-i', '--iparams',
4320                                      nargs='+',
4321                                      help='instrument parameter file, must be one'
4322                                           ' for every histogram'
4323                                      ),
4324
4325                         _args_kwargs('-p', '--phases',
4326                                      nargs='+',
4327                                      help='list of phases to add. phases are '
4328                                           'automatically associated with all '
4329                                           'histograms given.')]),
4330               "add": (add, [_args_kwargs('filename',
4331                                      help='the project file to open. Should end in .gpx'),
4332
4333                         _args_kwargs('-d', '--histograms',
4334                                      nargs='+',
4335                                      help='list of datafiles to add as histograms'),
4336                                     
4337                         _args_kwargs('-i', '--iparams',
4338                                      nargs='+',
4339                                      help='instrument parameter file, must be one'
4340                                           ' for every histogram'
4341                                      ),
4342                                     
4343                         _args_kwargs('-hf', '--histogramformat',
4344                                      help='format hint for histogram import. Applies to all'
4345                                           ' histograms'
4346                                      ),
4347
4348                         _args_kwargs('-p', '--phases',
4349                                      nargs='+',
4350                                      help='list of phases to add. phases are '
4351                                           'automatically associated with all '
4352                                           'histograms given.'),
4353
4354                         _args_kwargs('-pf', '--phaseformat',
4355                                      help='format hint for phase import. Applies to all'
4356                                           ' phases. Example: -pf CIF'
4357                                      ),
4358                                     
4359                         _args_kwargs('-l', '--histlist',
4360                                      nargs='+',
4361                                      help='list of histgram indices to associate with added'
4362                                           ' phases. If not specified, phases are'
4363                                           ' associated with all previously loaded'
4364                                           ' histograms. Example: -l 2 3 4')]),
4365                                           
4366               "dump": (dump, [_args_kwargs('-d', '--histograms',
4367                                     action='store_true',
4368                                     help='list histograms in files, overrides --raw'),
4369
4370                               _args_kwargs('-p', '--phases',
4371                                            action='store_true',
4372                                            help='list phases in files, overrides --raw'),
4373
4374                               _args_kwargs('-r', '--raw',
4375                                      action='store_true', help='dump raw file contents, default'),
4376
4377                               _args_kwargs('files', nargs='+')]),
4378
4379               "refine":
4380               (refine, [_args_kwargs('gpxfile', help='the project file to refine'),
4381                         _args_kwargs('refinements',
4382                                      help='JSON file of refinements to apply. if not present'
4383                                           ' refines file as-is',
4384                                      default=None,
4385                                      nargs='?')]),
4386
4387               "export": (export, [_args_kwargs('gpxfile',
4388                                                help='the project file from which to export'),
4389                                   _args_kwargs('phase', help='identifier of phase to export'),
4390                                   _args_kwargs('exportfile', help='the .cif file to export to')]),
4391               "browse": (IPyBrowse, [_args_kwargs('files', nargs='+',
4392                                                   help='list of files to browse')])}
4393
4394
4395def main():
4396    '''The command-line interface for calling GSASIIscriptable as a shell command,
4397    where it is expected to be called as::
4398
4399       python GSASIIscriptable.py <subcommand> <file.gpx> <options>
4400
4401    The following subcommands are defined:
4402
4403        * create, see :func:`create`
4404        * add, see :func:`add`
4405        * dump, see :func:`dump`
4406        * refine, see :func:`refine`
4407        * export, :func:`export`
4408        * browse, see :func:`IPyBrowse`
4409
4410    .. seealso::
4411        :func:`create`
4412        :func:`add`
4413        :func:`dump`
4414        :func:`refine`
4415        :func:`export`
4416        :func:`IPyBrowse`
4417    '''
4418    parser = argparse.ArgumentParser(description=
4419        "Use of "+os.path.split(__file__)[1]+" Allows GSAS-II actions from command line."
4420        )
4421    subs = parser.add_subparsers()
4422
4423    # Create all of the specified subparsers
4424    for name, (func, args) in subcommands.items():
4425        new_parser = subs.add_parser(name,help=commandhelp.get(name),
4426                                     description='Command "'+name+'" '+commandhelp.get(name))
4427        for listargs, kwds in args:
4428            new_parser.add_argument(*listargs, **kwds)
4429        new_parser.set_defaults(func=func)
4430
4431    # Parse and trigger subcommand
4432    result = parser.parse_args()
4433    result.func(result)
4434
4435if __name__ == '__main__':
4436    #fname='/tmp/corundum-template.gpx'
4437    #prj = G2Project(fname)
4438    main()
Note: See TracBrowser for help on using the repository browser.