source: trunk/GSASIIobj.py @ 4306

Last change on this file since 4306 was 4306, checked in by vondreele, 22 months ago

major doc edits

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 138.4 KB
Line 
1# -*- coding: utf-8 -*-
2#GSASIIobj - data objects for GSAS-II
3########### SVN repository information ###################
4# $Date: 2020-02-15 16:04:50 +0000 (Sat, 15 Feb 2020) $
5# $Author: vondreele $
6# $Revision: 4306 $
7# $URL: trunk/GSASIIobj.py $
8# $Id: GSASIIobj.py 4306 2020-02-15 16:04:50Z vondreele $
9########### SVN repository information ###################
10
11'''
12*GSASIIobj: Data objects*
13=========================
14
15This module defines and/or documents the data structures used in GSAS-II, as well
16as provides misc. support routines.
17
18.. _Constraints_table:
19
20.. index::
21   single: Constraints object description
22   single: Data object descriptions; Constraints
23
24Constraints Tree Item
25----------------------
26
27Constraints are stored in a dict, separated into groups.
28Note that parameter are named in the following pattern,
29p:h:<var>:n, where p is the phase number, h is the histogram number
30<var> is a variable name and n is the parameter number.
31If a parameter does not depend on a histogram or phase or is unnumbered, that
32number is omitted.
33Note that the contents of each dict item is a List where each element in the
34list is a :ref:`constraint definition objects <Constraint_definitions_table>`.
35The constraints in this form are converted in
36:func:`GSASIIstrIO.ProcessConstraints` to the form used in :mod:`GSASIImapvars`
37
38The keys in the Constraints dict are:
39
40.. tabularcolumns:: |l|p{4.5in}|
41
42==========  ====================================================
43  key         explanation
44==========  ====================================================
45Hist        This specifies a list of constraints on
46            histogram-related parameters,
47            which will be of form :h:<var>:n.
48HAP         This specifies a list of constraints on parameters
49            that are defined for every histogram in each phase
50            and are of form p:h:<var>:n.
51Phase       This specifies a list of constraints on phase
52            parameters,
53            which will be of form p::<var>:n.
54Global      This specifies a list of constraints on parameters
55            that are not tied to a histogram or phase and
56            are of form ::<var>:n
57==========  ====================================================
58
59.. _Constraint_definitions_table:
60
61.. index::
62   single: Constraint definition object description
63   single: Data object descriptions; Constraint Definition
64
65Each constraint is defined as an item in a list. Each constraint is of form::
66
67[[<mult1>, <var1>], [<mult2>, <var2>],..., <fixedval>, <varyflag>, <constype>]
68
69Where the variable pair list item containing two values [<mult>, <var>], where:
70
71  * <mult> is a multiplier for the constraint (float)
72  * <var> a :class:`G2VarObj` object (previously a str variable name of form
73      'p:h:name[:at]')
74
75Note that the last three items in the list play a special role:
76
77 * <fixedval> is the fixed value for a `constant equation` (``constype=c``)
78   constraint or is None. For a `New variable` (``constype=f``) constraint,
79   a variable name can be specified as a str (used for externally
80   generated constraints)
81 * <varyflag> is True or False for `New variable` (``constype=f``) constraints
82   or is None. This indicates if this variable should be refined.
83 * <constype> is one of four letters, 'e', 'c', 'h', 'f' that determines the type of constraint:
84
85    * 'e' defines a set of equivalent variables. Only the first variable is refined (if the
86      appropriate refine flag is set) and and all other equivalent variables in the list
87      are generated from that variable, using the appropriate multipliers.
88    * 'c' defines a constraint equation of form,
89      :math:`m_1 \\times var_1 + m_2 \\times var_2 + ... = c`
90    * 'h' defines a variable to hold (not vary). Any variable on this list is not varied,
91      even if its refinement flag is set. Only one [mult,var] pair is allowed in a hold
92      constraint and the mult value is ignored.
93      This is of particular value when needing to hold one or more variables where a
94      single flag controls a set of variables such as, coordinates,
95      the reciprocal metric tensor or anisotropic displacement parameter.
96    * 'f' defines a new variable (function) according to relationship
97      :math:`newvar = m_1 \\times var_1 + m_2 \\times var_2 + ...`
98
99.. _Covariance_table:
100
101.. index::
102   single: Covariance description
103   single: Data object descriptions; Covariance
104
105Covariance Tree Item
106--------------------
107
108The Covariance tree item has results from the last least-squares run. They
109are stored in a dict with these keys:
110
111.. tabularcolumns:: |l|l|p{4in}|
112
113=============  ===============  ====================================================
114  key            sub-key        explanation
115=============  ===============  ====================================================
116newCellDict    \                dict with lattice parameters computed by
117                                :func:`GSASIIstrMath.GetNewCellParms` (dict)
118title          \                Name of gpx file(?) (str)
119variables      \                Values for all N refined variables
120                                (list of float values, length N,
121                                ordered to match varyList)
122sig            \                Uncertainty values for all N refined variables
123                                (list of float values, length N,
124                                ordered to match varyList)
125varyList       \                List of directly refined variables
126                                (list of str values, length N)
127newAtomDict    \                dict with atom position values computed in
128                                :func:`GSASIIstrMath.ApplyXYZshifts` (dict)
129Rvals          \                R-factors, GOF, Marquardt value for last
130                                refinement cycle (dict)
131\              Nobs             Number of observed data points (int)
132\              Rwp              overall weighted profile R-factor (%, float)
133\              chisq            sum[w*(Iobs-Icalc)**2] for all data
134                                note this is not the reduced chi squared (float)
135\              lamMax           Marquardt value applied to Hessian diagonal
136                                (float)
137\              GOF              The goodness-of-fit, aka square root of
138                                the reduced chi squared. (float)
139covMatrix      \                The (NxN) covVariance matrix (np.array)
140=============  ===============  ====================================================
141
142.. _Phase_table:
143
144.. index::
145   single: Phase object description
146   single: Data object descriptions; Phase
147
148Phase Tree Items
149----------------
150
151Phase information is stored in the GSAS-II data tree as children of the
152Phases item in a dict with keys:
153
154.. tabularcolumns:: |l|l|p{4in}|
155
156==========  ===============  ====================================================
157  key         sub-key        explanation
158==========  ===============  ====================================================
159General         \            Overall information for the phase (dict)
160  \         3Dproj           projections for 3D pole distribution plots
161  \         AngleRadii       Default radius for each atom used to compute
162                             interatomic angles (list of floats)
163  \         AtomMass         List of masses for atoms
164  \         AtomPtrs         list of four locations (cx,ct,cs & cu) to use to pull info
165                             from the atom records (list)
166  \         AtomTypes        List of atom types
167  \         BondRadii        Default radius for each atom used to compute
168                             interatomic distances (list of floats)
169  \         Cell             List with 8 items: cell refinement flag (bool)
170                             a, b, c, (Angstrom, float)
171                             alpha, beta & gamma (degrees, float)
172                             volume (A^3, float)
173  \         Color            Colors for atoms (list of (r,b,g) triplets)
174  \         Compare          dict of polygon comparison parameters
175  \         Data plot type   data plot type ('Mustrain', 'Size' or
176                             'Preferred orientation') for powder data (str)
177  \         DisAglCtls       Dict with distance/angle search controls,
178                             which has keys 'Name', 'AtomTypes',
179                             'BondRadii', 'AngleRadii' which are as above
180                             except are possibly edited. Also contains
181                             'Factors', which is a 2 element list with
182                             a multiplier for bond and angle search range
183                             [typically (0.85,0.85)].
184  \         F000X            x-ray F(000) intensity (float)
185  \         F000N            neutron F(000) intensity (float)
186  \         Flip             dict of Charge flip controls
187  \         HydIds           dict geometrically generated hydrogen atoms
188  \         Isotope          dict of isotopes for each atom type
189  \         Isotopes         dict of scattering lengths for each isotope
190                             combination for each element in phase
191  \         MCSA controls    Monte Carlo-Simulated Annealing controls (dict)
192  \         Map              dict of map parameters
193  \         Mass             Mass of unit cell contents in g/mol
194  \         Modulated        bool True if phase modulated
195  \         Mydir            directory of current .gpx file (str)
196  \         Name             phase name (str)
197  \         Type             'nuclear' or 'macromolecular' for now (str)
198  \         NoAtoms          Number of atoms per unit cell of each type (dict)
199  \         POhkl            March-Dollase preferred orientation direction
200  \         Pawley dmin      maximum Q (as d-space) to use for Pawley
201                             extraction (float)
202  \         Pawley dmax      minimum Q (as d-space) to use for Pawley
203                             extraction (float)
204  \         Pawley neg wt    Restraint value for negative Pawley intensities
205                             (float)
206  \         SGData           Space group details as a :ref:`space group (SGData) object <SGData_table>`
207                             as defined in :func:`GSASIIspc.SpcGroup`.
208  \         SH Texture       dict of spherical harmonic preferred orientation
209                             parameters
210  \         Super            int dimension of super group (0,1 only)
211  \         Type             str phase type (e.g. 'nuclear')
212  \         Z                dict of atomic numbers for each atom type
213  \         doDysnomia       bool flag for max ent map modification via Dysnomia
214  \         doPawley         bool Flag for Pawley intensity extraction
215  \         vdWRadii         dict of van der Waals radii for each atom type
216ranId           \            unique random number Id for phase (int)
217pId             \            Phase Id number for current project (int).
218Atoms           \            Atoms in phase as a list of lists. The outer list
219                             is for each atom, the inner list contains varying
220                             items depending on the type of phase, see
221                             the :ref:`Atom Records <Atoms_table>` description.
222                             (list of lists)
223Drawing         \            Display parameters (dict)
224\           Atoms            A list of lists with an entry for each atom
225                             that is plotted.
226\           Plane            list Controls for contour density plane display
227\           Quaternion       Viewing quaternion (4 element np.array)
228\           Zclip            clipping distance in A (float)
229\           Zstep            Step to de/increase Z-clip (float)
230\           atomPtrs         positions of x, type, site sym, ADP flag in Draw Atoms (list)
231\           backColor        background for plot as and R,G,B triplet
232                             (default = [0, 0, 0], black).
233                             (list with three atoms)
234\           ballScale        Size of spheres in ball-and-stick display (float)
235\           bondList         dict with bonds
236\           bondRadius       Size of binds in A (float)
237\           cameraPos        Viewing position in A for plot (float)
238\           contourLevel     map contour level in e/A^3 (float)
239\           contourMax       float map contour maximum
240\           depthFog         True if use depthFog on plot - set currently as False (bool)
241\           ellipseProb      Probability limit for display of thermal
242                             ellipsoids in % (float).
243\           magMult          float multiplier for magnetic moment arrows
244\           mapSize          float x & y dimensions of contourmap (fixed internally)
245\           modelView        4,4 array from openGL drawing transofmation matrix
246\           oldxy            previous view point (list with two floats)
247\           radiusFactor     Distance ratio for searching for bonds. ? Bonds
248                             are located that are within r(Ra+Rb) and (Ra+Rb)/r
249                             where Ra and Rb are the atomic radii.
250\           selectedAtoms    List of selected atoms (list of int values)
251\           showABC          Flag to show view point triplet (bool). True=show.
252\           showHydrogen     Flag to control plotting of H atoms.
253\           showRigidBodies  Flag to highlight rigid body placement
254\           showSlice        flag to show contour map
255\           sizeH            Size ratio for H atoms (float)
256\           unitCellBox      Flag to control display of the unit cell.
257\           vdwScale         Multiplier of van der Waals radius for
258                             display of vdW spheres.
259\           viewDir          cartesian viewing direction (np.array with three
260                             elements)
261\           viewPoint        list of lists. First item in list is [x,y,z]
262                             in fractional coordinates for the center of
263                             the plot. Second item list of previous & current
264                             atom number viewed (may be [0,0])
265RBModels        \            Rigid body assignments (note Rigid body definitions
266                             are stored in their own main top-level tree entry.)
267RMC             \            dict RMCProfile & rmcfull controls
268Pawley ref      \            Pawley reflections
269Histograms      \            A dict of dicts. The key for the outer dict is
270                             the histograms tied to this phase. The inner
271                             dict contains the combined phase/histogram
272                             parameters for items such as scale factors,
273                             size and strain parameters. The following are the
274                             keys to the inner dict. (dict)
275\           Babinet          For protein crystallography. Dictionary with two
276                             entries, 'BabA', 'BabU'
277\           Extinction       Extinction parameter (list of float, bool)
278\           HStrain          Hydrostatic strain. List of two lists. The first is
279                             a list of the HStrain parameters (1, 2, 3, 4, or 6
280                             depending on unit cell), the second is a list of boolean
281                             refinement parameters (same length)
282\           Histogram        The name of the associated histogram (str)
283\           LeBail           Flag for LeBail extraction (bool)
284\           Mustrain         List of microstrain parameters, in order:
285
286                               0. Type, one of  u'isotropic', u'uniaxial',
287                                  u'generalized'
288                               1. Isotropic/uniaxial parameters - list of 3 floats
289                               2. Refinement flags - list of 3 bools
290                               3. Microstrain axis - list of 3 ints, [h, k, l]
291                               4. Generalized mustrain parameters - list of 2-6
292                                  floats, depending on space group
293                               5. Generalized refinement flags - list of bools,
294                                  corresponding to the parameters of (4)
295\           Pref.Ori.        Preferred Orientation. List of eight parameters.
296                             Items marked SH are only used for Spherical Harmonics.
297
298                               0. Type, 'MD' for March-Dollase or 'SH' for
299                                  Spherical Harmonics
300                               1. Value, float
301                               2. Refinement flag, bool
302                               3. Preferred direction, list of ints, [h, k, l]
303                               4. SH - number of terms, int
304                               5. SH - dict
305                               6. SH - list
306                               7. SH - float
307\           Scale            Phase fraction, list of [float, bool].
308\           Show             bool
309\           Size             List of crystallite size parameters, in order:
310   
311                               0. Type, one of  u'isotropic', u'uniaxial',
312                                  u'ellipsoidal'
313                               1. Isotropic/uniaxial parameters - list of 3 floats
314                               2. Refinement flags - list of 3 bools
315                               3. Size axis - list of 3 ints, [h, k, l]
316                               4. Ellipsoidal size parameters - list of 6
317                                  floats
318                               5. Ellipsoidal refinement flags - list of bools,
319                                  corresponding to the parameters of (4)
320\           Use              bool
321\           newLeBail        Whether to perform a new LeBail extraction
322MCSA            \            Monte-Carlo simulated annealing parameters (dict)
323\
324==========  ===============  ====================================================
325
326.. _RBData_table:
327
328.. index::
329   single: Rigid Body Data description
330   single: Data object descriptions; Rigid Body Data
331
332Rigid Body Objects
333------------------
334
335Rigid body descriptions are available for two types of rigid bodies: 'Vector'
336and 'Residue'. Vector rigid bodies are developed by a sequence of translations each
337with a refinable magnitude and Residue rigid bodies are described as Cartesian coordinates
338with defined refinable torsion angles.
339
340.. tabularcolumns:: |l|l|p{4in}|
341
342==========  ===============  ====================================================
343  key         sub-key        explanation
344==========  ===============  ====================================================
345Vector      RBId             vector rigid bodies (dict of dict)
346\           AtInfo           Drad, Color: atom drawing radius & color for each atom type (dict)
347\           RBname           Name assigned by user to rigid body (str)
348\           VectMag          vector magnitudes in A (list)
349\           rbXYZ            Cartesian coordinates for Vector rigid body (list of 3 float)
350\           rbRef            3 assigned reference atom nos. in rigid body for origin
351                             definition, use center of atoms flag (list of 3 int & 1 bool)
352\           VectRef          refinement flags for VectMag values (list of bool)
353\           rbTypes          Atom types for each atom in rigid body (list of str)
354\           rbVect           Cartesian vectors for each translation used to build rigid body (list of lists)
355\           useCount         Number of times rigid body is used in any structure (int)
356Residue     RBId             residue rigid bodies (dict of dict)
357\           AtInfo           Drad, Color: atom drawing radius & color for each atom type(dict)
358\           RBname           Name assigned by user to rigid body (str)
359\           rbXYZ            Cartesian coordinates for Residue rigid body (list of 3 float)
360\           rbTypes          Atom types for each atom in rigid body (list of str)
361\           atNames          Names of each atom in rigid body (e.g. C1,N2...) (list of str)
362\           rbRef            3 assigned reference atom nos. in rigid body for origin
363                             definition, use center of atoms flag (list of 3 int & 1 bool)
364\           rbSeq            Orig,Piv,angle,Riding (list): definition of internal rigid body
365                             torsion; origin atom (int), pivot atom (int), torsion angle (float),
366                             riding atoms (list of int)
367\           SelSeq           [int,int] used by SeqSizer to identify objects
368\           useCount         Number of times rigid body is used in any structure (int)
369RBIds           \            unique Ids generated upon creation of each rigid body (dict)
370\           Vector           Ids for each Vector rigid body (list)
371\           Residue          Ids for each Residue rigid body (list)
372==========  ===============  ====================================================
373
374.. _SGData_table:
375
376.. index::
377   single: Space Group Data description
378   single: Data object descriptions; Space Group Data
379
380Space Group Objects
381-------------------
382
383Space groups are interpreted by :func:`GSASIIspc.SpcGroup`
384and the information is placed in a SGdata object
385which is a dict with these keys. Magnetic ones are marked "mag"
386
387.. tabularcolumns:: |l|p{4.5in}|
388
389==========  ====================================================
390  key         explanation
391==========  ====================================================
392BNSlattsym  mag - BNS magnetic space group symbol and centering vector
393GenFlg      mag - list of symmetry generators indices
394GenSym      mag - list of names for each generator
395MagMom      mag - list of "time reversals" for each magnetic operator
396MagPtGp     mag  Magnetic point group symbol
397MagSpGrp    mag - Magnetic space group symbol
398OprNames    mag - names for each space group operation
399SGCen       Symmetry cell centering vectors. A (n,3) np.array
400            of centers. Will always have at least one row:
401            ``np.array([[0, 0, 0]])``
402SGFixed     bool only True if phase mported from a magnetic cif file
403            then the space group can not be changed by the user because
404            operator set from cif may be nonstandard
405SGGen       list of generators
406SGGray      True if space group is a gray group (incommensurate magnetic structures)
407SGInv       True if centrosymmetric, False if not (bool)
408SGLatt      Lattice centering type. Will be one of
409            P, A, B, C, I, F, R (str)
410SGLaue        one of the following 14 Laue classes:
411            -1, 2/m, mmm, 4/m, 4/mmm, 3R,
412            3mR, 3, 3m1, 31m, 6/m, 6/mmm, m3, m3m (str)
413SGOps       symmetry operations as a list of form
414            ``[[M1,T1], [M2,T2],...]``
415            where :math:`M_n` is a 3x3 np.array
416            and :math:`T_n` is a length 3 np.array.
417            Atom coordinates are transformed where the
418            Asymmetric unit coordinates [X is (x,y,z)]
419            are transformed using
420            :math:`X^\prime = M_n*X+T_n`
421SGPolax     Axes for space group polarity. Will be one of
422            '', 'x', 'y', 'x y', 'z', 'x z', 'y z',
423            'xyz'. In the case where axes are arbitrary
424            '111' is used (P 1, and ?).
425SGPtGrp     Point group f the space group
426SGUniq      unique axis if monoclinic. Will be
427            a, b, or c for monoclinic space groups.
428            Will be blank for non-monoclinic. (str)
429SGSpin      mag - list of spin flip operatiors (+1 or -1) for the space group operations
430SGSys       symmetry unit cell: type one of
431            'triclinic', 'monoclinic', 'orthorhombic',
432            'tetragonal', 'rhombohedral', 'trigonal',
433            'hexagonal', 'cubic' (str)
434SSGK1       list of superspace multipliers
435SpGrp       space group symbol (str)
436SpnFlp      mag - list of magnetic spin flips for every magnetic space group operator
437           
438==========  ====================================================
439
440.. _SSGData_table:
441
442.. index::
443   single: Superspace Group Data description
444   single: Data object descriptions; Superspace Group Data
445
446Superspace groups [3+1] are interpreted by :func:`GSASIIspc.SSpcGroup`
447and the information is placed in a SSGdata object
448which is a dict with these keys:
449
450.. tabularcolumns:: |l|p{4.5in}|
451
452==========  ====================================================
453  key         explanation
454==========  ====================================================
455SSpGrp      superspace group symbol extension to space group
456            symbol, accidental spaces removed
457SSGCen      4D cell centering vectors [0,0,0,0] at least
458SSGOps      4D symmetry operations as [M,T] so that M*x+T = x'
459==========  ====================================================
460
461
462.. _Atoms_table:
463
464.. index::
465   single: Atoms record description
466   single: Data object descriptions; Atoms record
467
468Atom Records
469------------
470
471If ``phasedict`` points to the phase information in the data tree, then
472atoms are contained in a list of atom records (list) in
473``phasedict['Atoms']``. Also needed to read atom information
474are four pointers, ``cx,ct,cs,cia = phasedict['General']['AtomPtrs']``,
475which define locations in the atom record, as shown below. Items shown are
476always present; additional ones for macromolecular phases are marked 'mm',
477and those for magnetic structures are marked 'mg'
478
479.. tabularcolumns:: |l|p{4.5in}|
480
481==============   ====================================================
482location         explanation
483==============   ====================================================
484ct-4              mm - residue number (str)
485ct-3              mm - residue name (e.g. ALA) (str)
486ct-2              mm - chain label (str)
487ct-1              atom label (str)
488ct                atom type (str)
489ct+1              refinement flags; combination of 'F', 'X', 'U', 'M' (str)
490cx,cx+1,cx+2      the x,y and z coordinates (3 floats)
491cx+3              site occupancy (float)
492cx+4,5,6          mg - atom magnetic moment along a,b,c in Bohr magnetons
493cs                site symmetry (str)
494cs+1              site multiplicity (int)
495cia               ADP flag: Isotropic ('I') or Anisotropic ('A')
496cia+1             Uiso (float)
497cia+2...cia+7     U11, U22, U33, U12, U13, U23 (6 floats)
498atom[cia+8]       unique atom identifier (int)
499
500==============   ====================================================
501
502.. _Drawing_atoms_table:
503
504.. index::
505   single: Drawing atoms record description
506   single: Data object descriptions; Drawing atoms record
507
508Drawing Atom Records
509--------------------
510
511If ``phasedict`` points to the phase information in the data tree, then
512drawing atoms are contained in a list of drawing atom records (list) in
513``phasedict['Drawing']['Atoms']``. Also needed to read atom information
514are four pointers, ``cx,ct,cs,ci = phasedict['Drawing']['AtomPtrs']``,
515which define locations in the atom record, as shown below. Items shown are
516always present; additional ones for macromolecular phases are marked 'mm',
517and those for magnetic structures are marked 'mg'
518
519.. tabularcolumns:: |l|p{4.5in}|
520
521==============   ====================================================
522location         explanation
523==============   ====================================================
524ct-4              mm - residue number (str)
525ct-3              mm - residue name (e.g. ALA) (str)
526ct-2              mm - chain label (str)
527ct-1              atom label (str)
528ct                atom type (str)
529cx,cx+1,cx+2      the x,y and z coordinates (3 floats)
530cx+3,4,5          mg - atom magnetic moment along a,b,c in Bohr magnetons
531cs-1              Sym Op symbol; sym. op number + unit cell id (e.g. '1,0,-1') (str)
532cs                atom drawing style; e.g. 'balls & sticks' (str)
533cs+1              atom label style (e.g. 'name') (str)
534cs+2              atom color (RBG triplet) (int)
535cs+3              ADP flag: Isotropic ('I') or Anisotropic ('A')
536cs+4              Uiso (float)
537cs+5...cs+11      U11, U22, U33, U12, U13, U23 (6 floats)
538ci                unique atom identifier; matches source atom Id in Atom Records (int)
539==============   ====================================================
540
541.. _Powder_table:
542
543.. index::
544   single: Powder data object description
545   single: Data object descriptions; Powder Data
546
547Powder Diffraction Tree Items
548-----------------------------
549
550Every powder diffraction histogram is stored in the GSAS-II data tree
551with a top-level entry named beginning with the string "PWDR ". The
552diffraction data for that information are directly associated with
553that tree item and there are a series of children to that item. The
554routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
555and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
556load this information into a dictionary where the child tree name is
557used as a key, and the information in the main entry is assigned
558a key of ``Data``, as outlined below.
559
560.. tabularcolumns:: |p{1in}|p{1in}|p{4in}|
561
562======================  ===============  ====================================================
563  key                      sub-key        explanation
564======================  ===============  ====================================================
565Comments                      \           Text strings extracted from the original powder
566                                          data header. These cannot be changed by the user;
567                                          it may be empty.
568Limits                       \            A list of two two element lists, as [[Ld,Hd],[L,H]]
569                                          where L and Ld are the current and default lowest
570                                          two-theta value to be used and
571                                          where H and Hd are the current and default highest
572                                          two-theta value to be used.
573Reflection Lists              \           A dict with an entry for each phase in the
574                                          histogram. The contents of each dict item
575                                          is a dict containing reflections, as described in
576                                          the :ref:`Powder Reflections <PowderRefl_table>`
577                                          description.
578Instrument Parameters (CW)    \           A list containing two dicts for constant wavelength
579                                          data where the possible keys in each dict are listed
580                                          below. The value for each item is a list containing
581                                          three values: the initial value, the current value
582                                          and a refinement flag which can have a value of
583                                          True, False or 0 where 0 indicates a value that
584                                          cannot be refined. The first and second
585                                          values are floats unless otherwise noted.
586                                          Items in the first dict are noted as [1]
587\                         Type            Histogram type (str) [1]:
588                                           * 'PXC' for constant wavelength x-ray
589                                           * 'PNC' for constant wavelength neutron
590\                         Bank            Data set number in a multidata file (usually 1)
591\                         Lam             Specifies a wavelength in Angstroms [1]
592\                         Lam1            Specifies the primary wavelength in
593                                          Angstrom, when an alpha1, alpha2
594                                          source is used [1]
595\                         Lam2            Specifies the secondary wavelength in
596                                          Angstrom, when an alpha1, alpha2
597                                          source is used [1]
598                          I(L2)/I(L1)     Ratio of Lam2 to Lam1 [1]
599\                         Zero            Two-theta zero correction in *degrees* [1]
600\                         Azimuth         Azimuthal setting angle for data recorded
601                                          with differing setting angles [1]
602\                         U, V, W         Cagliotti profile coefficients
603                                          for Gaussian instrumental broadening, where the
604                                          FWHM goes as
605                                          :math:`U \\tan^2\\theta + V \\tan\\theta + W` [1]
606\                         X, Y, Z         Cauchy (Lorentzian) instrumental broadening
607                                          coefficients [1]
608\                         SH/L            Variant of the Finger-Cox-Jephcoat asymmetric
609                                          peak broadening ratio. Note that this is the
610                                          sum of S/L and H/L where S is
611                                          sample height, H is the slit height and
612                                          L is the goniometer diameter. [1]
613\                         Polariz.        Polarization coefficient. [1]
614Instrument Parameters (TOF)   \           A list containing two dicts for neutron time-of-flight
615                                          (TOF) data where the possible keys in each dict are listed
616                                          below. The value for each item is a list containing
617                                          three values: the initial value, the current value
618                                          and a refinement flag which can have a value of
619                                          True, False or 0 where 0 indicates a value that
620                                          cannot be refined. The first and second
621                                          values are floats unless otherwise noted.
622                                          Items in the first dict are noted as [1]
623\                         Type            Histogram type (str) [1]:
624                                           * 'PNT' for time of flight neutron
625\                         Bank            Data set number in a multidata file
626\                         2-theta         Nominal scattering angle for the detector
627\                         fltPath         Total flight path source-sample-detector
628\                         Azimuth         Azimuth angle for detector righ hand rotation
629                                          from horizontal away from source
630\                         difC,difA,difB  Diffractometer constants for conversion of d-spacing to TOF
631                                          in microseconds
632\                         Zero            Zero point offset (microseconds)
633\                         alpha           Exponential rise profile coefficients
634\                         beta-0          Exponential decay profile coefficients
635                          beta-1
636                          beta-q
637\                         sig-0           Gaussian profile coefficients
638                          sig-1
639                          sig-2
640                          sig-q   
641\                         X,Y,Z           Lorentzian profile coefficients 
642wtFactor                      \           A weighting factor to increase or decrease
643                                          the leverage of data in the histogram (float).
644                                          A value of 1.0 weights the data with their
645                                          standard uncertainties and a larger value
646                                          increases the weighting of the data (equivalent
647                                          to decreasing the uncertainties).
648Sample Parameters             \           Specifies a dict with parameters that describe how
649                                          the data were collected, as listed
650                                          below. Refinable parameters are a list containing
651                                          a float and a bool, where the second value
652                                          specifies if the value is refined, otherwise
653                                          the value is a float unless otherwise noted.
654\                         Scale           The histogram scale factor (refinable)
655\                         Absorption      The sample absorption coefficient as
656                                          :math:`\\mu r` where r is the radius
657                                          (refinable). Only valid for Debye-Scherrer geometry.
658\                         SurfaceRoughA   Surface roughness parameter A as defined by
659                                          Surotti,J. Appl. Cryst, 5,325-331, 1972.(refinable -
660                                          only valid for Bragg-Brentano geometry)
661\                         SurfaceRoughB   Surface roughness parameter B (refinable -
662                                          only valid for Bragg-Brentano geometry)
663\                         DisplaceX,      Sample displacement from goniometer center
664                          DisplaceY       where Y is along the beam direction and
665                                          X is perpendicular. Units are :math:`\\mu m`
666                                          (refinable).
667\                         Phi, Chi,       Goniometer sample setting angles, in degrees.
668                          Omega
669\                         Gonio. radius   Radius of the diffractometer in mm
670\                         InstrName       A name for the instrument, used in preparing
671                                          a CIF (str).
672\                         Force,          Variables that describe how the measurement
673                          Temperature,    was performed. Not used directly in
674                          Humidity,       any computations.
675                          Pressure,
676                          Voltage
677\                         ranId           The random-number Id for the histogram
678                                          (same value as where top-level key is ranId)
679\                         Type            Type of diffraction data, may be 'Debye-Scherrer'
680                                          or 'Bragg-Brentano' (str).
681\                         Diffuse         not in use?
682hId                           \           The number assigned to the histogram when
683                                          the project is loaded or edited (can change)
684ranId                         \           A random number id for the histogram
685                                          that does not change
686Background                    \           The background is stored as a list with where
687                                          the first item in the list is list and the second
688                                          item is a dict. The list contains the background
689                                          function and its coefficients; the dict contains
690                                          Debye diffuse terms and background peaks.
691                                          (TODO: this needs to be expanded.)
692Data                          \           The data consist of a list of 6 np.arrays
693                                          containing in order:
694
695                                           0. the x-postions (two-theta in degrees),
696                                           1. the intensity values (Yobs),
697                                           2. the weights for each Yobs value
698                                           3. the computed intensity values (Ycalc)
699                                           4. the background values
700                                           5. Yobs-Ycalc
701======================  ===============  ====================================================
702
703.. _PowderRefl_table:
704
705.. index::
706   single: Powder reflection object description
707   single: Data object descriptions; Powder Reflections
708
709Powder Reflection Data Structure
710--------------------------------
711
712For every phase in a histogram, the ``Reflection Lists`` value is a dict
713one element of which is `'RefList'`, which is a np.array containing
714reflections. The columns in that array are documented below.
715
716==========  ====================================================
717  index         explanation
718==========  ====================================================
719 0,1,2       h,k,l (float)
720 3           multiplicity
721 4           d-space, Angstrom
722 5           pos, two-theta
723 6           sig, Gaussian width
724 7           gam, Lorenzian width
725 8           :math:`F_{obs}^2`
726 9           :math:`F_{calc}^2`
727 10          reflection phase, in degrees
728 11          intensity correction for reflection, this times
729             :math:`F_{obs}^2` or :math:`F_{calc}^2` gives Iobs or Icalc
730 12          Preferred orientation correction
731 13          Transmission (absorption correction)
732 14          Extinction correction
733==========  ====================================================
734
735.. _Xtal_table:
736
737.. index::
738   single: Single Crystal data object description
739   single: Data object descriptions; Single crystal data
740
741Single Crystal Tree Items
742-------------------------
743
744Every single crystal diffraction histogram is stored in the GSAS-II data tree
745with a top-level entry named beginning with the string "HKLF ". The
746diffraction data for that information are directly associated with
747that tree item and there are a series of children to that item. The
748routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
749and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
750load this information into a dictionary where the child tree name is
751used as a key, and the information in the main entry is assigned
752a key of ``Data``, as outlined below.
753
754.. tabularcolumns:: |l|l|p{4in}|
755
756======================  ===============  ====================================================
757  key                      sub-key        explanation
758======================  ===============  ====================================================
759Data                          \           A dict that contains the
760                                          reflection table,
761                                          as described in the
762                                          :ref:`Single Crystal Reflections
763                                          <XtalRefl_table>`
764                                          description.
765
766Instrument Parameters         \           A list containing two dicts where the possible
767                                          keys in each dict are listed below. The value
768                                          for most items is a list containing two values:
769                                          the initial value, the current value.
770                                          The first and second
771                                          values are floats unless otherwise noted.
772\                         Lam             Specifies a wavelength in Angstroms (two floats)
773\                         Type            Histogram type (two str values):
774                                           * 'SXC' for constant wavelength x-ray
775                                           * 'SNC' for constant wavelength neutron
776                                           * 'SNT' for time of flight neutron
777\                         InstrName       A name for the instrument, used in preparing
778                                          a CIF (str).
779
780wtFactor                      \           A weighting factor to increase or decrease
781                                          the leverage of data in the histogram (float).
782                                          A value of 1.0 weights the data with their
783                                          standard uncertainties and a larger value
784                                          increases the weighting of the data (equivalent
785                                          to decreasing the uncertainties).
786
787hId                           \           The number assigned to the histogram when
788                                          the project is loaded or edited (can change)
789ranId                         \           A random number id for the histogram
790                                          that does not change
791======================  ===============  ====================================================
792
793.. _XtalRefl_table:
794
795.. index::
796   single: Single Crystal reflection object description
797   single: Data object descriptions; Single Crystal Reflections
798
799Single Crystal Reflection Data Structure
800----------------------------------------
801
802For every single crystal a histogram, the ``'Data'`` item contains
803the structure factors as an np.array in item `'RefList'`.
804The columns in that array are documented below.
805
806==========  ====================================================
807  index         explanation
808==========  ====================================================
809 0,1,2       h,k,l (float)
810 3           multiplicity
811 4           d-space, Angstrom
812 5           :math:`F_{obs}^2`
813 6           :math:`\sigma(F_{obs}^2)`
814 7           :math:`F_{calc}^2`
815 8           :math:`F_{obs}^2T`
816 9           :math:`F_{calc}^2T`
817 10          reflection phase, in degrees
818 11          intensity correction for reflection, this times
819             :math:`F_{obs}^2` or :math:`F_{calc}^2`
820             gives Iobs or Icalc
821==========  ====================================================
822
823.. _Image_table:
824
825.. index::
826   image: Image data object description
827   image: Image object descriptions
828
829Image Data Structure
830--------------------
831
832Every 2-dimensional image is stored in the GSAS-II data tree
833with a top-level entry named beginning with the string "IMG ". The
834image data are directly associated with that tree item and there
835are a series of children to that item. The routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
836and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
837load this information into a dictionary where the child tree name is
838used as a key, and the information in the main entry is assigned
839a key of ``Data``, as outlined below.
840
841.. tabularcolumns:: |l|l|p{4in}|
842
843======================  ======================  ====================================================
844  key                      sub-key              explanation
845======================  ======================  ====================================================
846Comments                       \                Text strings extracted from the original image data
847                                                header or a metafile. These cannot be changed by
848                                                the user; it may be empty.
849Image Controls              azmthOff            (float) The offset to be applied to an azimuthal
850                                                value. Accomodates
851                                                detector orientations other than with the detector
852                                                X-axis
853                                                horizontal.
854\                           background image    (list:str,float) The name of a tree item ("IMG ...") that is to be subtracted
855                                                during image integration multiplied by value. It must have the same size/shape as
856                                                the integrated image. NB: value < 0 for subtraction.
857\                           calibrant           (str) The material used for determining the position/orientation
858                                                of the image. The data is obtained from :func:`ImageCalibrants`
859                                                and UserCalibrants.py (supplied by user).
860\                           calibdmin           (float) The minimum d-spacing used during the last calibration run.
861\                           calibskip           (int) The number of expected diffraction lines skipped during the last
862                                                calibration run.
863\                           center              (list:floats) The [X,Y] point in detector coordinates (mm) where the direct beam
864                                                strikes the detector plane as determined by calibration. This point
865                                                does not have to be within the limits of the detector boundaries.
866\                           centerAzm           (bool) If True then the azimuth reported for the integrated slice
867                                                of the image is at the center line otherwise it is at the leading edge.
868\                           color               (str) The name of the colormap used to display the image. Default = 'Paired'.
869\                           cutoff              (float) The minimum value of I/Ib for a point selected in a diffraction ring for
870                                                calibration calculations. See pixLimit for details as how point is found.
871\                           DetDepth            (float) Coefficient for penetration correction to distance; accounts for diffraction
872                                                ring offset at higher angles. Optionally determined by calibration.
873\                           DetDepthRef         (bool) If True then refine DetDepth during calibration/recalibration calculation.
874\                           distance            (float) The distance (mm) from sample to detector plane.
875\                           ellipses            (list:lists) Each object in ellipses is a list [center,phi,radii,color] where
876                                                center (list) is location (mm) of the ellipse center on the detector plane, phi is the
877                                                rotation of the ellipse minor axis from the x-axis, and radii are the minor & major
878                                                radii of the ellipse. If radii[0] is negative then parameters describe a hyperbola. Color
879                                                is the selected drawing color (one of 'b', 'g' ,'r') for the ellipse/hyperbola.
880\                           edgemin             (float) Not used;  parameter in EdgeFinder code.
881\                           fullIntegrate       (bool) If True then integrate over full 360 deg azimuthal range.
882\                           GonioAngles         (list:floats) The 'Omega','Chi','Phi' goniometer angles used for this image.
883                                                Required for texture calculations.
884\                           invert_x            (bool) If True display the image with the x-axis inverted.
885\                           invert_y            (bool) If True display the image with the y-axis inverted.
886\                           IOtth               (list:floats) The minimum and maximum 2-theta values to be used for integration.
887\                           LRazimuth           (list:floats) The minimum and maximum azimuth values to be used for integration.
888\                           Oblique             (list:float,bool) If True apply a detector absorption correction using the value to the
889                                                intensities obtained during integration.
890\                           outAzimuths         (int) The number of azimuth pie slices.
891\                           outChannels         (int) The number of 2-theta steps.
892\                           pixelSize           (list:ints) The X,Y dimensions (microns) of each pixel.
893\                           pixLimit            (int) A box in the image with 2*pixLimit+1 edges is searched to find the maximum.
894                                                This value (I) along with the minimum (Ib) in the box is reported by :func:`GSASIIimage.ImageLocalMax`
895                                                and subject to cutoff in :func:`GSASIIimage.makeRing`.
896                                                Locations are used to construct rings of points for calibration calcualtions.
897\                           PolaVal             (list:float,bool) If type='SASD' and if True, apply polarization correction to intensities from
898                                                integration using value.
899\                           rings               (list:lists) Each entry is [X,Y,dsp] where X & Y are lists of x,y coordinates around a
900                                                diffraction ring with the same d-spacing (dsp)
901\                           ring                (list) The x,y coordinates of the >5 points on an inner ring
902                                                selected by the user,
903\                           Range               (list) The minimum & maximum values of the image
904\                           rotation            (float) The angle between the x-axis and the vector about which the
905                                                detector is tilted. Constrained to -180 to 180 deg.
906\                           SampleShape         (str) Currently only 'Cylinder'. Sample shape for Debye-Scherrer experiments; used for absorption
907                                                calculations.
908\                           SampleAbs           (list: float,bool) Value of absorption coefficient for Debye-Scherrer experimnents, flag if True
909                                                to cause correction to be applied.
910\                           setDefault          (bool) If True the use the image controls values for all new images to be read. (might be removed)
911\                           setRings            (bool) If True then display all the selected x,y ring positions (vida supra rings) used in the calibration.
912\                           showLines           (bool) If True then isplay the integration limits to be used.
913\                           size                (list:int) The number of pixels on the image x & y axes
914\                           type                (str) One of 'PWDR', 'SASD' or 'REFL' for powder, small angle or reflectometry data, respectively.
915\                           tilt                (float) The angle the detector normal makes with the incident beam; range -90 to 90.
916\                           wavelength          (float) Tha radiation wavelength (Angstroms) as entered by the user (or someday obtained from the image header).
917
918Masks                       Arcs                (list: lists) Each entry [2-theta,[azimuth[0],azimuth[1]],thickness] describes an arc mask
919                                                to be excluded from integration
920\                           Frames              (list:lists) Each entry describes the x,y points (3 or more - mm) that describe a frame outside
921                                                of which is excluded from recalibration and integration. Only one frame is allowed.
922\                           Points              (list:lists) Each entry [x,y,radius] (mm) describes an excluded spot on the image to be excluded
923                                                from integration.
924\                           Polygons            (list:lists) Each entry is a list of 3+ [x,y] points (mm) that describe a polygon on the image
925                                                to be excluded from integration.
926\                           Rings               (list: lists) Each entry [2-theta,thickness] describes a ring mask
927                                                to be excluded from integration.
928\                           Thresholds          (list:[tuple,list]) [(Imin,Imax),[Imin,Imax]] This gives lower and upper limits for points on the image to be included
929                                                in integrsation. The tuple is the image intensity limits and the list are those set by the user.
930\                           SpotMask            dict: 'esdMul'(int) number of standard deviations above mean ring intensity to mask
931                                                'spotMask' (bool array) the spot mask for every pixel in image         
932
933Stress/Strain               Sample phi          (float) Sample rotation about vertical axis.
934\                           Sample z            (float) Sample translation from the calibration sample position (for Sample phi = 0)
935                                                These will be restricted by space group symmetry; result of strain fit refinement.
936\                           Type                (str) 'True' or 'Conventional': The strain model used for the calculation.
937\                           d-zero              (list:dict) Each item is for a diffraction ring on the image; all items are from the same phase
938                                                and are used to determine the strain tensor.
939                                                The dictionary items are:
940                                                'Dset': (float) True d-spacing for the diffraction ring; entered by the user.
941                                                'Dcalc': (float) Average calculated d-spacing determined from strain coeff.
942                                                'Emat': (list: float) The strain tensor elements e11, e12 & e22 (e21=e12, rest are 0)
943                                                'Esig': (list: float) Esds for Emat from fitting.
944                                                'pixLimit': (int) Search range to find highest point on ring for each data point
945                                                'cutoff': (float) I/Ib cutoff for searching.
946                                                'ImxyObs': (list: lists) [[X],[Y]] observed points to be used for strain calculations.
947                                                'ImtaObs': (list: lists) [[d],[azm]] transformed via detector calibration from ImxyObs.
948                                                'ImtaCalc': (list: lists [[d],[azm]] calculated d-spacing & azimuth from fit.
949
950======================  ======================  ====================================================
951
952.. _parmDict_table:
953
954.. index::
955   single: Parameter dictionary
956
957Parameter Dictionary
958-------------------------
959
960The parameter dictionary contains all of the variable parameters for the refinement.
961The dictionary keys are the name of the parameter (<phase>:<hist>:<name>:<atom>).
962It is prepared in two ways. When loaded from the tree
963(in :meth:`GSASIIdataGUI.GSASII.MakeLSParmDict` and
964:meth:`GSASIIIO.ExportBaseclass.loadParmDict`),
965the values are lists with two elements: ``[value, refine flag]``
966
967When loaded from the GPX file (in
968:func:`GSASIIstrMain.Refine` and :func:`GSASIIstrMain.SeqRefine`), the value in the
969dict is the actual parameter value (usually a float, but sometimes a
970letter or string flag value (such as I or A for iso/anisotropic).
971
972
973*Classes and routines*
974----------------------
975
976'''
977from __future__ import division, print_function
978import platform
979import re
980import imp
981import random as ran
982import sys
983import os.path as ospath
984if '2' in platform.python_version_tuple()[0]:
985    import cPickle
986else:
987    import pickle as cPickle
988import GSASIIpath
989import GSASIImath as G2mth
990import GSASIIspc as G2spc
991import numpy as np
992
993GSASIIpath.SetVersionNumber("$Revision: 4306 $")
994
995DefaultControls = {
996    'deriv type':'analytic Hessian',
997    'min dM/M':0.001,'shift factor':1.,'max cyc':3,'F**2':False,'SVDtol':1.e-6,
998    'UsrReject':{'minF/sig':0.,'MinExt':0.01,'MaxDF/F':100.,'MaxD':500.,'MinD':0.05},
999    'Copy2Next':False,'Reverse Seq':False,'HatomFix':False,
1000    'Author':'no name',
1001    'FreePrm1':'Sample humidity (%)',
1002    'FreePrm2':'Sample voltage (V)',
1003    'FreePrm3':'Applied load (MN)',
1004    'ShowCell':False,
1005    }
1006'''Values to be used as defaults for the initial contents of the ``Controls``
1007data tree item.
1008'''
1009def StripUnicode(string,subs='.'):
1010    '''Strip non-ASCII characters from strings
1011
1012    :param str string: string to strip Unicode characters from
1013    :param str subs: character(s) to place into string in place of each
1014      Unicode character. Defaults to '.'
1015
1016    :returns: a new string with only ASCII characters
1017    '''
1018    s = ''
1019    for c in string:
1020        if ord(c) < 128:
1021            s += c
1022        else:
1023            s += subs
1024    return s
1025#    return s.encode('ascii','replace')
1026
1027def MakeUniqueLabel(lbl,labellist):
1028    '''Make sure that every a label is unique against a list by adding
1029    digits at the end until it is not found in list.
1030
1031    :param str lbl: the input label
1032    :param list labellist: the labels that have already been encountered
1033    :returns: lbl if not found in labellist or lbl with ``_1-9`` (or
1034      ``_10-99``, etc.) appended at the end
1035    '''
1036    lbl = StripUnicode(lbl.strip(),'_')
1037    if not lbl: # deal with a blank label
1038        lbl = '_1'
1039    if lbl not in labellist:
1040        labellist.append(lbl)
1041        return lbl
1042    i = 1
1043    prefix = lbl
1044    if '_' in lbl:
1045        prefix = lbl[:lbl.rfind('_')]
1046        suffix = lbl[lbl.rfind('_')+1:]
1047        try:
1048            i = int(suffix)+1
1049        except: # suffix could not be parsed
1050            i = 1
1051            prefix = lbl
1052    while prefix+'_'+str(i) in labellist:
1053        i += 1
1054    else:
1055        lbl = prefix+'_'+str(i)
1056        labellist.append(lbl)
1057    return lbl
1058
1059PhaseIdLookup = {}
1060'''dict listing phase name and random Id keyed by sequential phase index as a str;
1061best to access this using :func:`LookupPhaseName`
1062'''
1063PhaseRanIdLookup = {}
1064'''dict listing phase sequential index keyed by phase random Id;
1065best to access this using :func:`LookupPhaseId`
1066'''
1067HistIdLookup = {}
1068'''dict listing histogram name and random Id, keyed by sequential histogram index as a str;
1069best to access this using :func:`LookupHistName`
1070'''
1071HistRanIdLookup = {}
1072'''dict listing histogram sequential index keyed by histogram random Id;
1073best to access this using :func:`LookupHistId`
1074'''
1075AtomIdLookup = {}
1076'''dict listing for each phase index as a str, the atom label and atom random Id,
1077keyed by atom sequential index as a str;
1078best to access this using :func:`LookupAtomLabel`
1079'''
1080AtomRanIdLookup = {}
1081'''dict listing for each phase the atom sequential index keyed by atom random Id;
1082best to access this using :func:`LookupAtomId`
1083'''
1084ShortPhaseNames = {}
1085'''a dict containing a possibly shortened and when non-unique numbered
1086version of the phase name. Keyed by the phase sequential index.
1087'''
1088ShortHistNames = {}
1089'''a dict containing a possibly shortened and when non-unique numbered
1090version of the histogram name. Keyed by the histogram sequential index.
1091'''
1092
1093#VarDesc = {}  # removed 1/30/19 BHT as no longer needed (I think)
1094#''' This dictionary lists descriptions for GSAS-II variables,
1095#as set in :func:`CompileVarDesc`. See that function for a description
1096#for how keys and values are written.
1097#'''
1098
1099reVarDesc = {}
1100''' This dictionary lists descriptions for GSAS-II variables where
1101keys are compiled regular expressions that will match the name portion
1102of a parameter name. Initialized in :func:`CompileVarDesc`.
1103'''
1104
1105reVarStep = {}
1106''' This dictionary lists the preferred step size for numerical
1107derivative computation w/r to a GSAS-II variable. Keys are compiled
1108regular expressions and values are the step size for that parameter.
1109Initialized in :func:`CompileVarDesc`.
1110'''
1111# create a default space group object for P1; N.B. fails when building documentation
1112try:
1113    P1SGData = G2spc.SpcGroup('P 1')[1] # data structure for default space group
1114except:
1115    pass
1116
1117def GetPhaseNames(fl):
1118    ''' Returns a list of phase names found under 'Phases' in GSASII gpx file
1119    NB: there is another one of these in GSASIIstrIO.py that uses the gpx filename
1120
1121    :param file fl: opened .gpx file
1122    :return: list of phase names
1123    '''
1124    PhaseNames = []
1125    while True:
1126        try:
1127            data = cPickle.load(fl)
1128        except EOFError:
1129            break
1130        datum = data[0]
1131        if 'Phases' == datum[0]:
1132            for datus in data[1:]:
1133                PhaseNames.append(datus[0])
1134    fl.seek(0)          #reposition file
1135    return PhaseNames
1136
1137def SetNewPhase(Name='New Phase',SGData=None,cell=None,Super=None):
1138    '''Create a new phase dict with default values for various parameters
1139
1140    :param str Name: Name for new Phase
1141
1142    :param dict SGData: space group data from :func:`GSASIIspc:SpcGroup`;
1143      defaults to data for P 1
1144
1145    :param list cell: unit cell parameter list; defaults to
1146      [1.0,1.0,1.0,90.,90,90.,1.]
1147
1148    '''
1149    if SGData is None: SGData = P1SGData
1150    if cell is None: cell=[1.0,1.0,1.0,90.,90.,90.,1.]
1151    phaseData = {
1152        'ranId':ran.randint(0,sys.maxsize),
1153        'General':{
1154            'Name':Name,
1155            'Type':'nuclear',
1156            'Modulated':False,
1157            'AtomPtrs':[3,1,7,9],
1158            'SGData':SGData,
1159            'Cell':[False,]+cell,
1160            'Pawley dmin':1.0,
1161            'Data plot type':'None',
1162            'SH Texture':{
1163                'Order':0,
1164                'Model':'cylindrical',
1165                'Sample omega':[False,0.0],
1166                'Sample chi':[False,0.0],
1167                'Sample phi':[False,0.0],
1168                'SH Coeff':[False,{}],
1169                'SHShow':False,
1170                'PFhkl':[0,0,1],
1171                'PFxyz':[0,0,1],
1172                'PlotType':'Pole figure',
1173                'Penalty':[['',],0.1,False,1.0]}},
1174        'Atoms':[],
1175        'Drawing':{},
1176        'Histograms':{},
1177        'Pawley ref':[],
1178        'RBModels':{},
1179        }
1180    if Super and Super.get('Use',False):
1181        phaseData['General'].update({'Modulated':True,'Super':True,'SuperSg':Super['ssSymb']})
1182        phaseData['General']['SSGData'] = G2spc.SSpcGroup(SGData,Super['ssSymb'])[1]
1183        phaseData['General']['SuperVec'] = [Super['ModVec'],False,Super['maxH']]
1184
1185    return phaseData
1186
1187def ReadCIF(URLorFile):
1188    '''Open a CIF, which may be specified as a file name or as a URL using PyCifRW
1189    (from James Hester).
1190    The open routine gets confused with DOS names that begin with a letter and colon
1191    "C:\dir\" so this routine will try to open the passed name as a file and if that
1192    fails, try it as a URL
1193
1194    :param str URLorFile: string containing a URL or a file name. Code will try first
1195      to open it as a file and then as a URL.
1196
1197    :returns: a PyCifRW CIF object.
1198    '''
1199    import CifFile as cif # PyCifRW from James Hester
1200
1201    # alternate approach:
1202    #import urllib
1203    #ciffile = 'file:'+urllib.pathname2url(filename)
1204
1205    try:
1206        fp = open(URLorFile,'r')
1207        cf = cif.ReadCif(fp)
1208        fp.close()
1209        return cf
1210    except IOError:
1211        return cif.ReadCif(URLorFile)
1212
1213def IndexAllIds(Histograms,Phases):
1214    '''Scan through the used phases & histograms and create an index
1215    to the random numbers of phases, histograms and atoms. While doing this,
1216    confirm that assigned random numbers are unique -- just in case lightning
1217    strikes twice in the same place.
1218
1219    Note: this code assumes that the atom random Id (ranId) is the last
1220    element each atom record.
1221
1222    This is called in three places (only): :func:`GSASIIstrIO.GetUsedHistogramsAndPhases`
1223    (which loads the histograms and phases from a GPX file),
1224    :meth:`~GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
1225    (which loads the histograms and phases from the data tree.) and
1226    :meth:`GSASIIconstrGUI.UpdateConstraints`
1227    (which displays & edits the constraints in a GUI)
1228
1229    TODO: do we need a lookup for rigid body variables?
1230    '''
1231    # process phases and atoms
1232    PhaseIdLookup.clear()
1233    PhaseRanIdLookup.clear()
1234    AtomIdLookup.clear()
1235    AtomRanIdLookup.clear()
1236    ShortPhaseNames.clear()
1237    for ph in Phases:
1238        cx,ct,cs,cia = Phases[ph]['General']['AtomPtrs']
1239        ranId = Phases[ph]['ranId']
1240        while ranId in PhaseRanIdLookup:
1241            # Found duplicate random Id! note and reassign
1242            print ("\n\n*** Phase "+str(ph)+" has repeated ranId. Fixing.\n")
1243            Phases[ph]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1244        pId = str(Phases[ph]['pId'])
1245        PhaseIdLookup[pId] = (ph,ranId)
1246        PhaseRanIdLookup[ranId] = pId
1247        shortname = ph  #[:10]
1248        while shortname in ShortPhaseNames.values():
1249            shortname = ph[:8] + ' ('+ pId + ')'
1250        ShortPhaseNames[pId] = shortname
1251        AtomIdLookup[pId] = {}
1252        AtomRanIdLookup[pId] = {}
1253        for iatm,at in enumerate(Phases[ph]['Atoms']):
1254            ranId = at[cia+8]
1255            while ranId in AtomRanIdLookup[pId]: # check for dups
1256                print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1257                at[cia+8] = ranId = ran.randint(0,sys.maxsize)
1258            AtomRanIdLookup[pId][ranId] = str(iatm)
1259            if Phases[ph]['General']['Type'] == 'macromolecular':
1260                label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1261            else:
1262                label = at[ct-1]
1263            AtomIdLookup[pId][str(iatm)] = (label,ranId)
1264    # process histograms
1265    HistIdLookup.clear()
1266    HistRanIdLookup.clear()
1267    ShortHistNames.clear()
1268    for hist in Histograms:
1269        ranId = Histograms[hist]['ranId']
1270        while ranId in HistRanIdLookup:
1271            # Found duplicate random Id! note and reassign
1272            print ("\n\n*** Histogram "+str(hist)+" has repeated ranId. Fixing.\n")
1273            Histograms[hist]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1274        hId = str(Histograms[hist]['hId'])
1275        HistIdLookup[hId] = (hist,ranId)
1276        HistRanIdLookup[ranId] = hId
1277        shortname = hist[:15]
1278        while shortname in ShortHistNames.values():
1279            shortname = hist[:11] + ' ('+ hId + ')'
1280        ShortHistNames[hId] = shortname
1281
1282def LookupAtomId(pId,ranId):
1283    '''Get the atom number from a phase and atom random Id
1284
1285    :param int/str pId: the sequential number of the phase
1286    :param int ranId: the random Id assigned to an atom
1287
1288    :returns: the index number of the atom (str)
1289    '''
1290    if not AtomRanIdLookup:
1291        raise Exception('Error: LookupAtomId called before IndexAllIds was run')
1292    if pId is None or pId == '':
1293        raise KeyError('Error: phase is invalid (None or blank)')
1294    pId = str(pId)
1295    if pId not in AtomRanIdLookup:
1296        raise KeyError('Error: LookupAtomId does not have phase '+pId)
1297    if ranId not in AtomRanIdLookup[pId]:
1298        raise KeyError('Error: LookupAtomId, ranId '+str(ranId)+' not in AtomRanIdLookup['+pId+']')
1299    return AtomRanIdLookup[pId][ranId]
1300
1301def LookupAtomLabel(pId,index):
1302    '''Get the atom label from a phase and atom index number
1303
1304    :param int/str pId: the sequential number of the phase
1305    :param int index: the index of the atom in the list of atoms
1306
1307    :returns: the label for the atom (str) and the random Id of the atom (int)
1308    '''
1309    if not AtomIdLookup:
1310        raise Exception('Error: LookupAtomLabel called before IndexAllIds was run')
1311    if pId is None or pId == '':
1312        raise KeyError('Error: phase is invalid (None or blank)')
1313    pId = str(pId)
1314    if pId not in AtomIdLookup:
1315        raise KeyError('Error: LookupAtomLabel does not have phase '+pId)
1316    if index not in AtomIdLookup[pId]:
1317        raise KeyError('Error: LookupAtomLabel, ranId '+str(index)+' not in AtomRanIdLookup['+pId+']')
1318    return AtomIdLookup[pId][index]
1319
1320def LookupPhaseId(ranId):
1321    '''Get the phase number and name from a phase random Id
1322
1323    :param int ranId: the random Id assigned to a phase
1324    :returns: the sequential Id (pId) number for the phase (str)
1325    '''
1326    if not PhaseRanIdLookup:
1327        raise Exception('Error: LookupPhaseId called before IndexAllIds was run')
1328    if ranId not in PhaseRanIdLookup:
1329        raise KeyError('Error: LookupPhaseId does not have ranId '+str(ranId))
1330    return PhaseRanIdLookup[ranId]
1331
1332def LookupPhaseName(pId):
1333    '''Get the phase number and name from a phase Id
1334
1335    :param int/str pId: the sequential assigned to a phase
1336    :returns:  (phase,ranId) where phase is the name of the phase (str)
1337      and ranId is the random # id for the phase (int)
1338    '''
1339    if not PhaseIdLookup:
1340        raise Exception('Error: LookupPhaseName called before IndexAllIds was run')
1341    if pId is None or pId == '':
1342        raise KeyError('Error: phase is invalid (None or blank)')
1343    pId = str(pId)
1344    if pId not in PhaseIdLookup:
1345        raise KeyError('Error: LookupPhaseName does not have index '+pId)
1346    return PhaseIdLookup[pId]
1347
1348def LookupHistId(ranId):
1349    '''Get the histogram number and name from a histogram random Id
1350
1351    :param int ranId: the random Id assigned to a histogram
1352    :returns: the sequential Id (hId) number for the histogram (str)
1353    '''
1354    if not HistRanIdLookup:
1355        raise Exception('Error: LookupHistId called before IndexAllIds was run')
1356    if ranId not in HistRanIdLookup:
1357        raise KeyError('Error: LookupHistId does not have ranId '+str(ranId))
1358    return HistRanIdLookup[ranId]
1359
1360def LookupHistName(hId):
1361    '''Get the histogram number and name from a histogram Id
1362
1363    :param int/str hId: the sequential assigned to a histogram
1364    :returns:  (hist,ranId) where hist is the name of the histogram (str)
1365      and ranId is the random # id for the histogram (int)
1366    '''
1367    if not HistIdLookup:
1368        raise Exception('Error: LookupHistName called before IndexAllIds was run')
1369    if hId is None or hId == '':
1370        raise KeyError('Error: histogram is invalid (None or blank)')
1371    hId = str(hId)
1372    if hId not in HistIdLookup:
1373        raise KeyError('Error: LookupHistName does not have index '+hId)
1374    return HistIdLookup[hId]
1375
1376def fmtVarDescr(varname):
1377    '''Return a string with a more complete description for a GSAS-II variable
1378
1379    :param str varname: A full G2 variable name with 2 or 3 or 4
1380       colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1381
1382    :returns: a string with the description
1383    '''
1384    s,l = VarDescr(varname)
1385    return s+": "+l
1386
1387def VarDescr(varname):
1388    '''Return two strings with a more complete description for a GSAS-II variable
1389
1390    :param str name: A full G2 variable name with 2 or 3 or 4
1391       colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1392
1393    :returns: (loc,meaning) where loc describes what item the variable is mapped
1394      (phase, histogram, etc.) and meaning describes what the variable does.
1395    '''
1396
1397    # special handling for parameter names without a colon
1398    # for now, assume self-defining
1399    if varname.find(':') == -1:
1400        return "Global",varname
1401
1402    l = getVarDescr(varname)
1403    if not l:
1404        return ("invalid variable name ("+str(varname)+")!"),""
1405#        return "invalid variable name!",""
1406
1407    if not l[-1]:
1408        l[-1] = "(variable needs a definition! Set it in CompileVarDesc)"
1409
1410    if len(l) == 3:         #SASD variable name!
1411        s = 'component:'+l[1]
1412        return s,l[-1]
1413    s = ""
1414    if l[0] is not None and l[1] is not None: # HAP: keep short
1415        if l[2] == "Scale": # fix up ambigous name
1416            l[5] = "Phase fraction"
1417        if l[0] == '*':
1418            lbl = 'Seq. ref.'
1419        else:
1420            lbl = ShortPhaseNames.get(l[0],'? #'+str(l[0]))
1421        if l[1] == '*':
1422            hlbl = 'Seq. ref.'
1423        else:
1424            hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1425        if hlbl[:4] == 'HKLF':
1426            hlbl = 'Xtl='+hlbl[5:]
1427        elif hlbl[:4] == 'PWDR':
1428            hlbl = 'Pwd='+hlbl[5:]
1429        else:
1430            hlbl = 'Hist='+hlbl
1431        s = "Ph="+str(lbl)+" * "+str(hlbl)
1432    else:
1433        if l[2] == "Scale": # fix up ambigous name: must be scale factor, since not HAP
1434            l[5] = "Scale factor"
1435        if l[2] == 'Back': # background parameters are "special", alas
1436            s = 'Hist='+ShortHistNames.get(l[1],'? #'+str(l[1]))
1437            l[-1] += ' #'+str(l[3])
1438        elif l[4] is not None: # rigid body parameter or modulation parm
1439            lbl = ShortPhaseNames.get(l[0],'phase?')
1440            if 'RB' in l[2]:    #rigid body parm
1441                s = "Res #"+str(l[3])+" body #"+str(l[4])+" in "+str(lbl)
1442            else: #modulation parm
1443                s = 'Atom %s wave %s in %s'%(LookupAtomLabel(l[0],l[3])[0],l[4],lbl)
1444        elif l[3] is not None: # atom parameter,
1445            lbl = ShortPhaseNames.get(l[0],'phase?')
1446            try:
1447                albl = LookupAtomLabel(l[0],l[3])[0]
1448            except KeyError:
1449                albl = 'Atom?'
1450            s = "Atom "+str(albl)+" in "+str(lbl)
1451        elif l[0] == '*':
1452            s = "All phases "
1453        elif l[0] is not None:
1454            lbl = ShortPhaseNames.get(l[0],'phase?')
1455            s = "Phase "+str(lbl)
1456        elif l[1] == '*':
1457            s = 'All hists'
1458        elif l[1] is not None:
1459            hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1460            if hlbl[:4] == 'HKLF':
1461                hlbl = 'Xtl='+hlbl[5:]
1462            elif hlbl[:4] == 'PWDR':
1463                hlbl = 'Pwd='+hlbl[5:]
1464            else:
1465                hlbl = 'Hist='+hlbl
1466            s = str(hlbl)
1467    if not s:
1468        s = 'Global'
1469    return s,l[-1]
1470
1471def getVarDescr(varname):
1472    '''Return a short description for a GSAS-II variable
1473
1474    :param str name: A full G2 variable name with 2 or 3 or 4
1475       colons (<p>:<h>:name[:<a1>][:<a2>])
1476
1477    :returns: a six element list as [`p`,`h`,`name`,`a1`,`a2`,`description`],
1478      where `p`, `h`, `a1`, `a2` are str values or `None`, for the phase number,
1479      the histogram number and the atom number; `name` will always be
1480      a str; and `description` is str or `None`.
1481      If the variable name is incorrectly formed (for example, wrong
1482      number of colons), `None` is returned instead of a list.
1483    '''
1484    l = varname.split(':')
1485    if len(l) == 2:     #SASD parameter name
1486        return varname,l[0],getDescr(l[1])
1487    if len(l) == 3:
1488        l += [None,None]
1489    elif len(l) == 4:
1490        l += [None]
1491    elif len(l) != 5:
1492        return None
1493    for i in (0,1,3,4):
1494        if l[i] == "":
1495            l[i] = None
1496    l += [getDescr(l[2])]
1497    return l
1498
1499def CompileVarDesc():
1500    '''Set the values in the variable lookup tables
1501    (:attr:`reVarDesc` and :attr:`reVarStep`).
1502    This is called in :func:`getDescr` and :func:`getVarStep` so this
1503    initialization is always done before use.
1504
1505    Note that keys may contain regular expressions, where '[xyz]'
1506    matches 'x' 'y' or 'z' (equivalently '[x-z]' describes this as range
1507    of values). '.*' matches any string. For example::
1508
1509    'AUiso':'Atomic isotropic displacement parameter',
1510
1511    will match variable ``'p::AUiso:a'``.
1512    If parentheses are used in the key, the contents of those parentheses can be
1513    used in the value, such as::
1514
1515    'AU([123][123])':'Atomic anisotropic displacement parameter U\\1',
1516
1517    will match ``AU11``, ``AU23``,.. and `U11`, `U23` etc will be displayed
1518    in the value when used.
1519
1520    '''
1521    if reVarDesc: return # already done
1522    for key,value in {
1523        # derived or other sequential vars
1524        '([abc])$' : 'Lattice parameter, \\1, from Ai and Djk', # N.B. '$' prevents match if any characters follow
1525        u'\u03B1' : u'Lattice parameter, \u03B1, from Ai and Djk',
1526        u'\u03B2' : u'Lattice parameter, \u03B2, from Ai and Djk',
1527        u'\u03B3' : u'Lattice parameter, \u03B3, from Ai and Djk',
1528        # ambiguous, alas:
1529        'Scale' : 'Phase or Histogram scale factor',
1530        # Phase vars (p::<var>)
1531        'A([0-5])' : ('Reciprocal metric tensor component \\1',1e-5),
1532        '[vV]ol' : 'Unit cell volume', # probably an error that both upper and lower case are used
1533        # Atom vars (p::<var>:a)
1534        'dA([xyz])$' : ('change to atomic coordinate, \\1',1e-6),
1535        'A([xyz])$' : '\\1 fractional atomic coordinate',
1536        'AUiso':('Atomic isotropic displacement parameter',1e-4),
1537        'AU([123][123])':('Atomic anisotropic displacement parameter U\\1',1e-4),
1538        'Afrac': ('Atomic site fraction parameter',1e-5),
1539        'Amul': 'Atomic site multiplicity value',
1540        'AM([xyz])$' : 'Atomic magnetic moment parameter, \\1',
1541        # Hist & Phase (HAP) vars (p:h:<var>)
1542        'Back': 'Background term',
1543        'BkPkint;(.*)':'Background peak #\\1 intensity',
1544        'BkPkpos;(.*)':'Background peak #\\1 position',
1545        'BkPksig;(.*)':'Background peak #\\1 Gaussian width',
1546        'BkPkgam;(.*)':'Background peak #\\1 Cauchy width',
1547        'Bab([AU])': 'Babinet solvent scattering coef. \\1',
1548        'D([123][123])' : 'Anisotropic strain coef. \\1',
1549        'Extinction' : 'Extinction coef.',
1550        'MD' : 'March-Dollase coef.',
1551        'Mustrain;.*' : 'Microstrain coef.',
1552        'Size;.*' : 'Crystallite size value',
1553        'eA$' : 'Cubic mustrain value',
1554        'Ep$' : 'Primary extinction',
1555        'Es$' : 'Secondary type II extinction',
1556        'Eg$' : 'Secondary type I extinction',
1557        'Flack' : 'Flack parameter',
1558        'TwinFr' : 'Twin fraction',
1559        #Histogram vars (:h:<var>)
1560        'Absorption' : 'Absorption coef.',
1561        'Displace([XY])' : ('Debye-Scherrer sample displacement \\1',0.1),
1562        'Lam' : ('Wavelength',1e-6),
1563        'I\(L2\)\/I\(L1\)' : ('Ka2/Ka1 intensity ratio',0.001),
1564        'Polariz\.' : ('Polarization correction',1e-3),
1565        'SH/L' : ('FCJ peak asymmetry correction',1e-4),
1566        '([UVW])$' : ('Gaussian instrument broadening \\1',1e-5),
1567        '([XYZ])$' : ('Cauchy instrument broadening \\1',1e-5),
1568        'Zero' : 'Debye-Scherrer zero correction',
1569        'Shift' : 'Bragg-Brentano sample displ.',
1570        'SurfRoughA' : 'Bragg-Brenano surface roughness A',
1571        'SurfRoughB' : 'Bragg-Brenano surface roughness B',
1572        'Transparency' : 'Bragg-Brentano sample tranparency',
1573        'DebyeA' : 'Debye model amplitude',
1574        'DebyeR' : 'Debye model radius',
1575        'DebyeU' : 'Debye model Uiso',
1576        'RBV.*' : 'Vector rigid body parameter',
1577        'RBR.*' : 'Residue rigid body parameter',
1578        'RBRO([aijk])' : 'Residue rigid body orientation parameter',
1579        'RBRP([xyz])' : 'Residue rigid body position parameter',
1580        'RBRTr;.*' : 'Residue rigid body torsion parameter',
1581        'RBR([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
1582        'constr([0-9]*)' : 'Parameter from constraint',
1583        # supersymmetry parameters  p::<var>:a:o 'Flen','Fcent'?
1584        'mV([0-2])$' : 'Modulation vector component \\1',
1585        'Fsin'  :   'Sin site fraction modulation',
1586        'Fcos'  :   'Cos site fraction modulation',
1587        'Fzero'  :   'Crenel function offset',      #may go away
1588        'Fwid'   :   'Crenel function width',
1589        'Tmin'   :   'ZigZag/Block min location',
1590        'Tmax'   :   'ZigZag/Block max location',
1591        '([XYZ])max': 'ZigZag/Block max value for \\1',
1592        '([XYZ])sin'  : 'Sin position wave for \\1',
1593        '([XYZ])cos'  : 'Cos position wave for \\1',
1594        'U([123][123])sin$' :  'Sin thermal wave for U\\1',
1595        'U([123][123])cos$' :  'Cos thermal wave for U\\1',
1596        'M([XYZ])sin$' :  'Sin mag. moment wave for \\1',
1597        'M([XYZ])cos$' :  'Cos mag. moment wave for \\1',
1598        # PDF peak parms (l:<var>;l = peak no.)
1599        'PDFpos'  : 'PDF peak position',
1600        'PDFmag'  : 'PDF peak magnitude',
1601        'PDFsig'  : 'PDF peak std. dev.',
1602        # SASD vars (l:<var>;l = component)
1603        'Aspect ratio' : 'Particle aspect ratio',
1604        'Length' : 'Cylinder length',
1605        'Diameter' : 'Cylinder/disk diameter',
1606        'Thickness' : 'Disk thickness',
1607        'Shell thickness' : 'Multiplier to get inner(<1) or outer(>1) sphere radius',
1608        'Dist' : 'Interparticle distance',
1609        'VolFr' : 'Dense scatterer volume fraction',
1610        'epis' : 'Sticky sphere epsilon',
1611        'Sticky' : 'Stickyness',
1612        'Depth' : 'Well depth',
1613        'Width' : 'Well width',
1614        'Volume' : 'Particle volume',
1615        'Radius' : 'Sphere/cylinder/disk radius',
1616        'Mean' : 'Particle mean radius',
1617        'StdDev' : 'Standard deviation in Mean',
1618        'G$': 'Guinier prefactor',
1619        'Rg$': 'Guinier radius of gyration',
1620        'B$': 'Porod prefactor',
1621        'P$': 'Porod power',
1622        'Cutoff': 'Porod cutoff',
1623        'PkInt': 'Bragg peak intensity',
1624        'PkPos': 'Bragg peak position',
1625        'PkSig': 'Bragg peak sigma',
1626        'PkGam': 'Bragg peak gamma',
1627        'e([12][12])' : 'strain tensor e\1',   # strain vars e11, e22, e12
1628        'Dcalc': 'Calc. d-spacing',
1629        'Back$': 'background parameter',
1630        'pos$': 'peak position',
1631        'int$': 'peak intensity',
1632        'WgtFrac':'phase weight fraction',
1633        'alpha':'TOF profile term',
1634        'beta-[01q]':'TOF profile term',
1635        'sig-[012q]':'TOF profile term',
1636        'dif[ABC]':'TOF to d-space calibration',
1637        'C\([0-9]*,[0-9]*\)' : 'spherical harmonics preferred orientation coef.',
1638        }.items():
1639        if len(value) == 2:
1640            #VarDesc[key] = value[0]
1641            reVarDesc[re.compile(key)] = value[0]
1642            reVarStep[re.compile(key)] = value[1]
1643        else:
1644            #VarDesc[key] = value
1645            reVarDesc[re.compile(key)] = value
1646
1647def removeNonRefined(parmList):
1648    '''Remove items from variable list that are not refined and should not
1649    appear as options for constraints
1650
1651    :param list parmList: a list of strings of form "p:h:VAR:a" where
1652      VAR is the variable name
1653
1654    :returns: a list after removing variables where VAR matches a
1655      entry in local variable NonRefinedList
1656    '''
1657    NonRefinedList = ['Omega','Type','Chi','Phi', 'Azimuth','Gonio. radius',
1658                          'Lam1','Lam2','Back','Temperature','Pressure',
1659                          'FreePrm1','FreePrm2','FreePrm3',
1660                          'Source','nPeaks','LeBail','newLeBail','Bank',
1661                          'nDebye', #'',
1662                    ]
1663    return [prm for prm in parmList if prm.split(':')[2] not in NonRefinedList]
1664       
1665def getDescr(name):
1666    '''Return a short description for a GSAS-II variable
1667
1668    :param str name: The descriptive part of the variable name without colons (:)
1669
1670    :returns: a short description or None if not found
1671    '''
1672
1673    CompileVarDesc() # compile the regular expressions, if needed
1674    for key in reVarDesc:
1675        m = key.match(name)
1676        if m:
1677            reVarDesc[key]
1678            return m.expand(reVarDesc[key])
1679    return None
1680
1681def getVarStep(name,parmDict=None):
1682    '''Return a step size for computing the derivative of a GSAS-II variable
1683
1684    :param str name: A complete variable name (with colons, :)
1685    :param dict parmDict: A dict with parameter values or None (default)
1686
1687    :returns: a float that should be an appropriate step size, either from
1688      the value supplied in :func:`CompileVarDesc` or based on the value for
1689      name in parmDict, if supplied. If not found or the value is zero,
1690      a default value of 1e-5 is used. If parmDict is None (default) and
1691      no value is provided in :func:`CompileVarDesc`, then None is returned.
1692    '''
1693    CompileVarDesc() # compile the regular expressions, if needed
1694    for key in reVarStep:
1695        m = key.match(name)
1696        if m:
1697            return reVarStep[key]
1698    if parmDict is None: return None
1699    val = parmDict.get(key,0.0)
1700    if abs(val) > 0.05:
1701        return abs(val)/1000.
1702    else:
1703        return 1e-5
1704
1705def GenWildCard(varlist):
1706    '''Generate wildcard versions of G2 variables. These introduce '*'
1707    for a phase, histogram or atom number (but only for one of these
1708    fields) but only when there is more than one matching variable in the
1709    input variable list. So if the input is this::
1710
1711      varlist = ['0::AUiso:0', '0::AUiso:1', '1::AUiso:0']
1712
1713    then the output will be this::
1714
1715       wildList = ['*::AUiso:0', '0::AUiso:*']
1716
1717    :param list varlist: an input list of GSAS-II variable names
1718      (such as 0::AUiso:0)
1719
1720    :returns: wildList, the generated list of wild card variable names.
1721    '''
1722    wild = []
1723    for i in (0,1,3):
1724        currentL = varlist[:]
1725        while currentL:
1726            item1 = currentL.pop(0)
1727            i1splt = item1.split(':')
1728            if i >= len(i1splt): continue
1729            if i1splt[i]:
1730                nextL = []
1731                i1splt[i] = '[0-9]+'
1732                rexp = re.compile(':'.join(i1splt))
1733                matchlist = [item1]
1734                for nxtitem in currentL:
1735                    if rexp.match(nxtitem):
1736                        matchlist += [nxtitem]
1737                    else:
1738                        nextL.append(nxtitem)
1739                if len(matchlist) > 1:
1740                    i1splt[i] = '*'
1741                    wild.append(':'.join(i1splt))
1742                currentL = nextL
1743    return wild
1744
1745def LookupWildCard(varname,varlist):
1746    '''returns a list of variable names from list varname
1747    that match wildcard name in varname
1748
1749    :param str varname: a G2 variable name containing a wildcard
1750      (such as \*::var)
1751    :param list varlist: the list of all variable names used in
1752      the current project
1753    :returns: a list of matching GSAS-II variables (may be empty)
1754    '''
1755    rexp = re.compile(varname.replace('*','[0-9]+'))
1756    return sorted([var for var in varlist if rexp.match(var)])
1757
1758
1759def _lookup(dic,key):
1760    '''Lookup a key in a dictionary, where None returns an empty string
1761    but an unmatched key returns a question mark. Used in :class:`G2VarObj`
1762    '''
1763    if key is None:
1764        return ""
1765    elif key == "*":
1766        return "*"
1767    else:
1768        return dic.get(key,'?')
1769
1770def SortVariables(varlist):
1771    '''Sorts variable names in a sensible manner
1772    '''
1773    def cvnnums(var):
1774        v = []
1775        for i in var.split(':'):
1776            if i == '':
1777                v.append(-1)
1778                continue
1779            try:
1780                v.append(int(i))
1781            except:
1782                v.append(i)
1783        return v
1784    return sorted(varlist,key=cvnnums)
1785
1786class G2VarObj(object):
1787    '''Defines a GSAS-II variable either using the phase/atom/histogram
1788    unique Id numbers or using a character string that specifies
1789    variables by phase/atom/histogram number (which can change).
1790    Note that :func:`LoadID` should be used to (re)load the current Ids
1791    before creating or later using the G2VarObj object.
1792
1793    This can store rigid body variables, but does not translate the residue # and
1794    body # to/from random Ids
1795
1796    A :class:`G2VarObj` object can be created with a single parameter:
1797
1798    :param str/tuple varname: a single value can be used to create a :class:`G2VarObj`
1799      object. If a string, it must be of form "p:h:var" or "p:h:var:a", where
1800
1801     * p is the phase number (which may be left blank or may be '*' to indicate all phases);
1802     * h is the histogram number (which may be left blank or may be '*' to indicate all histograms);
1803     * a is the atom number (which may be left blank in which case the third colon is omitted).
1804       The atom number can be specified as '*' if a phase number is specified (not as '*').
1805       For rigid body variables, specify a will be a string of form "residue:body#"
1806
1807      Alternately a single tuple of form (Phase,Histogram,VarName,AtomID) can be used, where
1808      Phase, Histogram, and AtomID are None or are ranId values (or one can be '*')
1809      and VarName is a string. Note that if Phase is '*' then the AtomID is an atom number.
1810      For a rigid body variables, AtomID is a string of form "residue:body#".
1811
1812    If four positional arguments are supplied, they are:
1813
1814    :param str/int phasenum: The number for the phase (or None or '*')
1815    :param str/int histnum: The number for the histogram (or None or '*')
1816    :param str varname: a single value can be used to create a :class:`G2VarObj`
1817    :param str/int atomnum: The number for the atom (or None or '*')
1818
1819    '''
1820    IDdict = {}
1821    IDdict['phases'] = {}
1822    IDdict['hists'] = {}
1823    IDdict['atoms'] = {}
1824    def __init__(self,*args):
1825        self.phase = None
1826        self.histogram = None
1827        self.name = ''
1828        self.atom = None
1829        if len(args) == 1 and (type(args[0]) is list or type(args[0]) is tuple) and len(args[0]) == 4:
1830            # single arg with 4 values
1831            self.phase,self.histogram,self.name,self.atom = args[0]
1832        elif len(args) == 1 and ':' in args[0]:
1833            #parse a string
1834            lst = args[0].split(':')
1835            if lst[0] == '*':
1836                self.phase = '*'
1837                if len(lst) > 3:
1838                    self.atom = lst[3]
1839                self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
1840            elif lst[1] == '*':
1841                self.histogram = '*'
1842                self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
1843            else:
1844                self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
1845                self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
1846                if len(lst) == 4:
1847                    if lst[3] == '*':
1848                        self.atom = '*'
1849                    else:
1850                        self.atom = AtomIdLookup[lst[0]].get(lst[3],[None,None])[1]
1851                elif len(lst) == 5:
1852                    self.atom = lst[3]+":"+lst[4]
1853                elif len(lst) == 3:
1854                    pass
1855                else:
1856                    raise Exception("Too many colons in var name "+str(args[0]))
1857            self.name = lst[2]
1858        elif len(args) == 4:
1859            if args[0] == '*':
1860                self.phase = '*'
1861                self.atom = args[3]
1862            else:
1863                self.phase = PhaseIdLookup.get(str(args[0]),[None,None])[1]
1864                if args[3] == '*':
1865                    self.atom = '*'
1866                elif args[0] is not None:
1867                    self.atom = AtomIdLookup[args[0]].get(str(args[3]),[None,None])[1]
1868            if args[1] == '*':
1869                self.histogram = '*'
1870            else:
1871                self.histogram = HistIdLookup.get(str(args[1]),[None,None])[1]
1872            self.name = args[2]
1873        else:
1874            raise Exception("Incorrectly called GSAS-II parameter name")
1875
1876        #print "DEBUG: created ",self.phase,self.histogram,self.name,self.atom
1877
1878    def __str__(self):
1879        return self.varname()
1880
1881    def varname(self):
1882        '''Formats the GSAS-II variable name as a "traditional" GSAS-II variable
1883        string (p:h:<var>:a) or (p:h:<var>)
1884
1885        :returns: the variable name as a str
1886        '''
1887        a = ""
1888        if self.phase == "*":
1889            ph = "*"
1890            if self.atom:
1891                a = ":" + str(self.atom)
1892        else:
1893            ph = _lookup(PhaseRanIdLookup,self.phase)
1894            if self.atom == '*':
1895                a = ':*'
1896            elif self.atom:
1897                if ":" in str(self.atom):
1898                    a = ":" + str(self.atom)
1899                elif ph in AtomRanIdLookup:
1900                    a = ":" + AtomRanIdLookup[ph].get(self.atom,'?')
1901                else:
1902                    a = ":?"
1903        if self.histogram == "*":
1904            hist = "*"
1905        else:
1906            hist = _lookup(HistRanIdLookup,self.histogram)
1907        s = (ph + ":" + hist + ":" + str(self.name)) + a
1908        return s
1909
1910    def __repr__(self):
1911        '''Return the detailed contents of the object
1912        '''
1913        s = "<"
1914        if self.phase == '*':
1915            s += "Phases: all; "
1916            if self.atom is not None:
1917                if ":" in str(self.atom):
1918                    s += "Rigid body" + str(self.atom) + "; "
1919                else:
1920                    s += "Atom #" + str(self.atom) + "; "
1921        elif self.phase is not None:
1922            ph =  _lookup(PhaseRanIdLookup,self.phase)
1923            s += "Phase: rId=" + str(self.phase) + " (#"+ ph + "); "
1924            if self.atom == '*':
1925                s += "Atoms: all; "
1926            elif ":" in str(self.atom):
1927                s += "Rigid body" + str(self.atom) + "; "
1928            elif self.atom is not None:
1929                s += "Atom rId=" + str(self.atom)
1930                if ph in AtomRanIdLookup:
1931                    s += " (#" + AtomRanIdLookup[ph].get(self.atom,'?') + "); "
1932                else:
1933                    s += " (#? -- not found!); "
1934        if self.histogram == '*':
1935            s += "Histograms: all; "
1936        elif self.histogram is not None:
1937            hist = _lookup(HistRanIdLookup,self.histogram)
1938            s += "Histogram: rId=" + str(self.histogram) + " (#"+ hist + "); "
1939        s += 'Variable name="' + str(self.name) + '">'
1940        return s+" ("+self.varname()+")"
1941
1942    def __eq__(self, other):
1943        if type(other) is type(self):
1944            return (self.phase == other.phase and
1945                    self.histogram == other.histogram and
1946                    self.name == other.name and
1947                    self.atom == other.atom)
1948        return False
1949
1950    def _show(self):
1951        'For testing, shows the current lookup table'
1952        print ('phases'+ self.IDdict['phases'])
1953        print ('hists'+ self.IDdict['hists'])
1954        print ('atomDict'+ self.IDdict['atoms'])
1955
1956#==========================================================================
1957def SetDefaultSample():
1958    'Fills in default items for the Sample dictionary for Debye-Scherrer & SASD'
1959    return {
1960        'InstrName':'',
1961        'ranId':ran.randint(0,sys.maxsize),
1962        'Scale':[1.0,True],'Type':'Debye-Scherrer','Absorption':[0.0,False],
1963        'DisplaceX':[0.0,False],'DisplaceY':[0.0,False],'Diffuse':[],
1964        'Temperature':300.,'Pressure':0.1,'Time':0.0,
1965        'FreePrm1':0.,'FreePrm2':0.,'FreePrm3':0.,
1966        'Gonio. radius':200.0,
1967        'Omega':0.0,'Chi':0.0,'Phi':0.0,'Azimuth':0.0,
1968#SASD items
1969        'Materials':[{'Name':'vacuum','VolFrac':1.0,},{'Name':'vacuum','VolFrac':0.0,}],
1970        'Thick':1.0,'Contrast':[0.0,0.0],       #contrast & anomalous contrast
1971        'Trans':1.0,                            #measured transmission
1972        'SlitLen':0.0,                          #Slit length - in Q(A-1)
1973        }
1974######################################################################
1975class ImportBaseclass(object):
1976    '''Defines a base class for the reading of input files (diffraction
1977    data, coordinates,...). See :ref:`Writing a Import Routine<import_routines>`
1978    for an explanation on how to use a subclass of this class.
1979    '''
1980    class ImportException(Exception):
1981        '''Defines an Exception that is used when an import routine hits an expected error,
1982        usually in .Reader.
1983
1984        Good practice is that the Reader should define a value in self.errors that
1985        tells the user some information about what is wrong with their file.
1986        '''
1987        pass
1988
1989    UseReader = True  # in __init__ set value of self.UseReader to False to skip use of current importer
1990    def __init__(self,formatName,longFormatName=None,
1991                 extensionlist=[],strictExtension=False,):
1992        self.formatName = formatName # short string naming file type
1993        if longFormatName: # longer string naming file type
1994            self.longFormatName = longFormatName
1995        else:
1996            self.longFormatName = formatName
1997        # define extensions that are allowed for the file type
1998        # for windows, remove any extensions that are duplicate, as case is ignored
1999        if sys.platform == 'windows' and extensionlist:
2000            extensionlist = list(set([s.lower() for s in extensionlist]))
2001        self.extensionlist = extensionlist
2002        # If strictExtension is True, the file will not be read, unless
2003        # the extension matches one in the extensionlist
2004        self.strictExtension = strictExtension
2005        self.errors = ''
2006        self.warnings = ''
2007        self.SciPy = False          #image reader needed scipy
2008        # used for readers that will use multiple passes to read
2009        # more than one data block
2010        self.repeat = False
2011        self.selections = []
2012        self.repeatcount = 0
2013        self.readfilename = '?'
2014        self.scriptable = False
2015        #print 'created',self.__class__
2016
2017    def ReInitialize(self):
2018        'Reinitialize the Reader to initial settings'
2019        self.errors = ''
2020        self.warnings = ''
2021        self.SciPy = False          #image reader needed scipy
2022        self.repeat = False
2023        self.repeatcount = 0
2024        self.readfilename = '?'
2025
2026
2027#    def Reader(self, filename, filepointer, ParentFrame=None, **unused):
2028#        '''This method must be supplied in the child class to read the file.
2029#        if the read fails either return False or raise an Exception
2030#        preferably of type ImportException.
2031#        '''
2032#        #start reading
2033#        raise ImportException("Error occurred while...")
2034#        self.errors += "Hint for user on why the error occur
2035#        return False # if an error occurs
2036#        return True # if read OK
2037
2038    def ExtensionValidator(self, filename):
2039        '''This methods checks if the file has the correct extension
2040       
2041        :returns:
2042       
2043          * False if this filename will not be supported by this reader (only
2044            when strictExtension is True)
2045          * True if the extension matches the list supplied by the reader
2046          * None if the reader allows un-registered extensions
2047         
2048        '''
2049        if filename:
2050            ext = ospath.splitext(filename)[1]
2051            if not ext and self.strictExtension: return False
2052            for ext in self.extensionlist:               
2053                if sys.platform == 'windows':
2054                    if filename.lower().endswith(ext): return True
2055                else:
2056                    if filename.endswith(ext): return True
2057        if self.strictExtension:
2058            return False
2059        else:
2060            return None
2061
2062    def ContentsValidator(self, filename):
2063        '''This routine will attempt to determine if the file can be read
2064        with the current format.
2065        This will typically be overridden with a method that
2066        takes a quick scan of [some of]
2067        the file contents to do a "sanity" check if the file
2068        appears to match the selected format.
2069        the file must be opened here with the correct format (binary/text)
2070        '''
2071        #filepointer.seek(0) # rewind the file pointer
2072        return True
2073
2074    def CIFValidator(self, filepointer):
2075        '''A :meth:`ContentsValidator` for use to validate CIF files.
2076        '''
2077        filepointer.seek(0)
2078        for i,l in enumerate(filepointer):
2079            if i >= 1000: return True
2080            '''Encountered only blank lines or comments in first 1000
2081            lines. This is unlikely, but assume it is CIF anyway, since we are
2082            even less likely to find a file with nothing but hashes and
2083            blank lines'''
2084            line = l.strip()
2085            if len(line) == 0: # ignore blank lines
2086                continue
2087            elif line.startswith('#'): # ignore comments
2088                continue
2089            elif line.startswith('data_'): # on the right track, accept this file
2090                return True
2091            else: # found something invalid
2092                self.errors = 'line '+str(i+1)+' contains unexpected data:\n'
2093                if all([ord(c) < 128 and ord(c) != 0 for c in str(l)]): # show only if ASCII
2094                    self.errors += '  '+str(l)
2095                else:
2096                    self.errors += '  (binary)'
2097                self.errors += '\n  Note: a CIF should only have blank lines or comments before'
2098                self.errors += '\n        a data_ statement begins a block.'
2099                return False
2100
2101######################################################################
2102class ImportPhase(ImportBaseclass):
2103    '''Defines a base class for the reading of files with coordinates
2104
2105    Objects constructed that subclass this (in import/G2phase_*.py etc.) will be used
2106    in :meth:`GSASIIdataGUI.GSASII.OnImportPhase` and in
2107    :func:`GSASIIscriptable.import_generic`.
2108    See :ref:`Writing a Import Routine<import_routines>`
2109    for an explanation on how to use this class.
2110
2111    '''
2112    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2113        strictExtension=False,):
2114        # call parent __init__
2115        ImportBaseclass.__init__(self,formatName,longFormatName,
2116            extensionlist,strictExtension)
2117        self.Phase = None # a phase must be created with G2IO.SetNewPhase in the Reader
2118        self.Constraints = None
2119
2120######################################################################
2121class ImportStructFactor(ImportBaseclass):
2122    '''Defines a base class for the reading of files with tables
2123    of structure factors.
2124
2125    Structure factors are read with a call to :meth:`GSASIIdataGUI.GSASII.OnImportSfact`
2126    which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`, which calls
2127    methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2128    :meth:`Reader`.
2129
2130    See :ref:`Writing a Import Routine<import_routines>`
2131    for an explanation on how to use import classes in general. The specifics
2132    for reading a structure factor histogram require that
2133    the ``Reader()`` routine in the import
2134    class need to do only a few things: It
2135    should load :attr:`RefDict` item ``'RefList'`` with the reflection list,
2136    and set :attr:`Parameters` with the instrument parameters
2137    (initialized with :meth:`InitParameters` and set with :meth:`UpdateParameters`).
2138    '''
2139    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2140        strictExtension=False,):
2141        ImportBaseclass.__init__(self,formatName,longFormatName,
2142            extensionlist,strictExtension)
2143
2144        # define contents of Structure Factor entry
2145        self.Parameters = []
2146        'self.Parameters is a list with two dicts for data parameter settings'
2147        self.InitParameters()
2148        self.RefDict = {'RefList':[],'FF':{},'Super':0}
2149        self.Banks = []             #for multi bank data (usually TOF)
2150        '''self.RefDict is a dict containing the reflection information, as read from the file.
2151        Item 'RefList' contains the reflection information. See the
2152        :ref:`Single Crystal Reflection Data Structure<XtalRefl_table>`
2153        for the contents of each row. Dict element 'FF'
2154        contains the form factor values for each element type; if this entry
2155        is left as initialized (an empty list) it will be initialized as needed later.
2156        '''
2157    def ReInitialize(self):
2158        'Reinitialize the Reader to initial settings'
2159        ImportBaseclass.ReInitialize(self)
2160        self.InitParameters()
2161        self.Banks = []             #for multi bank data (usually TOF)
2162        self.RefDict = {'RefList':[],'FF':{},'Super':0}
2163
2164    def InitParameters(self):
2165        'initialize the instrument parameters structure'
2166        Lambda = 0.70926
2167        HistType = 'SXC'
2168        self.Parameters = [{'Type':[HistType,HistType], # create the structure
2169                            'Lam':[Lambda,Lambda]
2170                            }, {}]
2171        'Parameters is a list with two dicts for data parameter settings'
2172
2173    def UpdateParameters(self,Type=None,Wave=None):
2174        'Revise the instrument parameters'
2175        if Type is not None:
2176            self.Parameters[0]['Type'] = [Type,Type]
2177        if Wave is not None:
2178            self.Parameters[0]['Lam'] = [Wave,Wave]
2179
2180######################################################################
2181class ImportPowderData(ImportBaseclass):
2182    '''Defines a base class for the reading of files with powder data.
2183
2184    Objects constructed that subclass this (in import/G2pwd_*.py etc.) will be used
2185    in :meth:`GSASIIdataGUI.GSASII.OnImportPowder` and in
2186    :func:`GSASIIscriptable.import_generic`.
2187    See :ref:`Writing a Import Routine<import_routines>`
2188    for an explanation on how to use this class.
2189    '''
2190    def __init__(self,formatName,longFormatName=None,
2191        extensionlist=[],strictExtension=False,):
2192        ImportBaseclass.__init__(self,formatName,longFormatName,
2193            extensionlist,strictExtension)
2194        self.clockWd = None  # used in TOF
2195        self.ReInitialize()
2196
2197    def ReInitialize(self):
2198        'Reinitialize the Reader to initial settings'
2199        ImportBaseclass.ReInitialize(self)
2200        self.powderentry = ['',None,None] #  (filename,Pos,Bank)
2201        self.powderdata = [] # Powder dataset
2202        '''A powder data set is a list with items [x,y,w,yc,yb,yd]:
2203                np.array(x), # x-axis values
2204                np.array(y), # powder pattern intensities
2205                np.array(w), # 1/sig(intensity)^2 values (weights)
2206                np.array(yc), # calc. intensities (zero)
2207                np.array(yb), # calc. background (zero)
2208                np.array(yd), # obs-calc profiles
2209        '''
2210        self.comments = []
2211        self.idstring = ''
2212        self.Sample = SetDefaultSample() # default sample parameters
2213        self.Controls = {}  # items to be placed in top-level Controls
2214        self.GSAS = None     # used in TOF
2215        self.repeat_instparm = True # Should a parm file be
2216        #                             used for multiple histograms?
2217        self.instparm = None # name hint from file of instparm to use
2218        self.instfile = '' # full path name to instrument parameter file
2219        self.instbank = '' # inst parm bank number
2220        self.instmsg = ''  # a label that gets printed to show
2221                           # where instrument parameters are from
2222        self.numbanks = 1
2223        self.instdict = {} # place items here that will be transferred to the instrument parameters
2224        self.pwdparms = {} # place parameters that are transferred directly to the tree
2225                           # here (typically from an existing GPX file)
2226######################################################################
2227class ImportSmallAngleData(ImportBaseclass):
2228    '''Defines a base class for the reading of files with small angle data.
2229    See :ref:`Writing a Import Routine<import_routines>`
2230    for an explanation on how to use this class.
2231    '''
2232    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2233        strictExtension=False,):
2234
2235        ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2236            strictExtension)
2237        self.ReInitialize()
2238
2239    def ReInitialize(self):
2240        'Reinitialize the Reader to initial settings'
2241        ImportBaseclass.ReInitialize(self)
2242        self.smallangleentry = ['',None,None] #  (filename,Pos,Bank)
2243        self.smallangledata = [] # SASD dataset
2244        '''A small angle data set is a list with items [x,y,w,yc,yd]:
2245                np.array(x), # x-axis values
2246                np.array(y), # powder pattern intensities
2247                np.array(w), # 1/sig(intensity)^2 values (weights)
2248                np.array(yc), # calc. intensities (zero)
2249                np.array(yd), # obs-calc profiles
2250                np.array(yb), # preset bkg
2251        '''
2252        self.comments = []
2253        self.idstring = ''
2254        self.Sample = SetDefaultSample()
2255        self.GSAS = None     # used in TOF
2256        self.clockWd = None  # used in TOF
2257        self.numbanks = 1
2258        self.instdict = {} # place items here that will be transferred to the instrument parameters
2259
2260######################################################################
2261class ImportReflectometryData(ImportBaseclass):
2262    '''Defines a base class for the reading of files with reflectometry data.
2263    See :ref:`Writing a Import Routine<import_routines>`
2264    for an explanation on how to use this class.
2265    '''
2266    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2267        strictExtension=False,):
2268
2269        ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2270            strictExtension)
2271        self.ReInitialize()
2272
2273    def ReInitialize(self):
2274        'Reinitialize the Reader to initial settings'
2275        ImportBaseclass.ReInitialize(self)
2276        self.reflectometryentry = ['',None,None] #  (filename,Pos,Bank)
2277        self.reflectometrydata = [] # SASD dataset
2278        '''A small angle data set is a list with items [x,y,w,yc,yd]:
2279                np.array(x), # x-axis values
2280                np.array(y), # powder pattern intensities
2281                np.array(w), # 1/sig(intensity)^2 values (weights)
2282                np.array(yc), # calc. intensities (zero)
2283                np.array(yd), # obs-calc profiles
2284                np.array(yb), # preset bkg
2285        '''
2286        self.comments = []
2287        self.idstring = ''
2288        self.Sample = SetDefaultSample()
2289        self.GSAS = None     # used in TOF
2290        self.clockWd = None  # used in TOF
2291        self.numbanks = 1
2292        self.instdict = {} # place items here that will be transferred to the instrument parameters
2293
2294######################################################################
2295class ImportPDFData(ImportBaseclass):
2296    '''Defines a base class for the reading of files with PDF G(R) data.
2297    See :ref:`Writing a Import Routine<import_routines>`
2298    for an explanation on how to use this class.
2299    '''
2300    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2301        strictExtension=False,):
2302
2303        ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2304            strictExtension)
2305        self.ReInitialize()
2306
2307    def ReInitialize(self):
2308        'Reinitialize the Reader to initial settings'
2309        ImportBaseclass.ReInitialize(self)
2310        self.pdfentry = ['',None,None] #  (filename,Pos,Bank)
2311        self.pdfdata = [] # PDF G(R) dataset
2312        '''A pdf g(r) data set is a list with items [x,y]:
2313                np.array(x), # r-axis values
2314                np.array(y), # pdf g(r)
2315        '''
2316        self.comments = []
2317        self.idstring = ''
2318        self.numbanks = 1
2319
2320######################################################################
2321class ImportImage(ImportBaseclass):
2322    '''Defines a base class for the reading of images
2323
2324    Images are read in only these places:
2325
2326      * Initial reading is typically done from a menu item
2327        with a call to :meth:`GSASIIdataGUI.GSASII.OnImportImage`
2328        which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`. That calls
2329        methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2330        :meth:`Reader`. This returns a list of reader objects for each read image.
2331        Also used in :func:`GSASIIscriptable.import_generic`.
2332
2333      * Images are read alternatively in :func:`GSASIIIO.ReadImages`, which puts image info
2334        directly into the data tree.
2335
2336      * Images are reloaded with :func:`GSASIIIO.GetImageData`.
2337
2338    When reading an image, the ``Reader()`` routine in the ImportImage class
2339    should set:
2340
2341      * :attr:`Comments`: a list of strings (str),
2342      * :attr:`Npix`: the number of pixels in the image (int),
2343      * :attr:`Image`: the actual image as a numpy array (np.array)
2344      * :attr:`Data`: a dict defining image parameters (dict). Within this dict the following
2345        data items are needed:
2346
2347         * 'pixelSize': size of each pixel in microns (such as ``[200.,200.]``.
2348         * 'wavelength': wavelength in Angstoms.
2349         * 'distance': distance of detector from sample in cm.
2350         * 'center': uncalibrated center of beam on detector (such as ``[204.8,204.8]``.
2351         * 'size': size of image (such as ``[2048,2048]``).
2352         * 'ImageTag': image number or other keyword used to retrieve image from
2353           a multi-image data file (defaults to ``1`` if not specified).
2354         * 'sumfile': holds sum image file name if a sum was produced from a multi image file
2355
2356    optional data items:
2357
2358      * :attr:`repeat`: set to True if there are additional images to
2359        read in the file, False otherwise
2360      * :attr:`repeatcount`: set to the number of the image.
2361
2362    Note that the above is initialized with :meth:`InitParameters`.
2363    (Also see :ref:`Writing a Import Routine<import_routines>`
2364    for an explanation on how to use import classes in general.)
2365    '''
2366    def __init__(self,formatName,longFormatName=None,extensionlist=[],
2367        strictExtension=False,):
2368        ImportBaseclass.__init__(self,formatName,longFormatName,
2369            extensionlist,strictExtension)
2370        self.InitParameters()
2371
2372    def ReInitialize(self):
2373        'Reinitialize the Reader to initial settings -- not used at present'
2374        ImportBaseclass.ReInitialize(self)
2375        self.InitParameters()
2376
2377    def InitParameters(self):
2378        'initialize the instrument parameters structure'
2379        self.Comments = ['No comments']
2380        self.Data = {}
2381        self.Npix = 0
2382        self.Image = None
2383        self.repeat = False
2384        self.repeatcount = 1
2385        self.sumfile = ''
2386
2387    def LoadImage(self,ParentFrame,imagefile,imagetag=None):
2388        '''Optionally, call this after reading in an image to load it into the tree.
2389        This saves time by preventing a reread of the same information.
2390        '''
2391        if ParentFrame:
2392            ParentFrame.ImageZ = self.Image   # store the image for plotting
2393            ParentFrame.oldImagefile = imagefile # save the name of the last image file read
2394            ParentFrame.oldImageTag = imagetag   # save the tag of the last image file read
2395
2396#################################################################################################
2397# shortcut routines
2398exp = np.exp
2399sind = sin = s = lambda x: np.sin(x*np.pi/180.)
2400cosd = cos = c = lambda x: np.cos(x*np.pi/180.)
2401tand = tan = t = lambda x: np.tan(x*np.pi/180.)
2402sqrt = sq = lambda x: np.sqrt(x)
2403pi = lambda: np.pi
2404class ExpressionObj(object):
2405    '''Defines an object with a user-defined expression, to be used for
2406    secondary fits or restraints. Object is created null, but is changed
2407    using :meth:`LoadExpression`. This contains only the minimum
2408    information that needs to be stored to save and load the expression
2409    and how it is mapped to GSAS-II variables.
2410    '''
2411    def __init__(self):
2412        self.expression = ''
2413        'The expression as a text string'
2414        self.assgnVars = {}
2415        '''A dict where keys are label names in the expression mapping to a GSAS-II
2416        variable. The value a G2 variable name.
2417        Note that the G2 variable name may contain a wild-card and correspond to
2418        multiple values.
2419        '''
2420        self.freeVars = {}
2421        '''A dict where keys are label names in the expression mapping to a free
2422        parameter. The value is a list with:
2423
2424         * a name assigned to the parameter
2425         * a value for to the parameter and
2426         * a flag to determine if the variable is refined.
2427        '''
2428        self.depVar = None
2429
2430        self.lastError = ('','')
2431        '''Shows last encountered error in processing expression
2432        (list of 1-3 str values)'''
2433
2434        self.distance_dict  = None  # to be used for defining atom phase/symmetry info
2435        self.distance_atoms = None  # to be used for defining atom distances
2436
2437    def LoadExpression(self,expr,exprVarLst,varSelect,varName,varValue,varRefflag):
2438        '''Load the expression and associated settings into the object. Raises
2439        an exception if the expression is not parsed, if not all functions
2440        are defined or if not all needed parameter labels in the expression
2441        are defined.
2442
2443        This will not test if the variable referenced in these definitions
2444        are actually in the parameter dictionary. This is checked when the
2445        computation for the expression is done in :meth:`SetupCalc`.
2446
2447        :param str expr: the expression
2448        :param list exprVarLst: parameter labels found in the expression
2449        :param dict varSelect: this will be 0 for Free parameters
2450          and non-zero for expression labels linked to G2 variables.
2451        :param dict varName: Defines a name (str) associated with each free parameter
2452        :param dict varValue: Defines a value (float) associated with each free parameter
2453        :param dict varRefflag: Defines a refinement flag (bool)
2454          associated with each free parameter
2455        '''
2456        self.expression = expr
2457        self.compiledExpr = None
2458        self.freeVars = {}
2459        self.assgnVars = {}
2460        for v in exprVarLst:
2461            if varSelect[v] == 0:
2462                self.freeVars[v] = [
2463                    varName.get(v),
2464                    varValue.get(v),
2465                    varRefflag.get(v),
2466                    ]
2467            else:
2468                self.assgnVars[v] = varName[v]
2469        self.CheckVars()
2470
2471    def EditExpression(self,exprVarLst,varSelect,varName,varValue,varRefflag):
2472        '''Load the expression and associated settings from the object into
2473        arrays used for editing.
2474
2475        :param list exprVarLst: parameter labels found in the expression
2476        :param dict varSelect: this will be 0 for Free parameters
2477          and non-zero for expression labels linked to G2 variables.
2478        :param dict varName: Defines a name (str) associated with each free parameter
2479        :param dict varValue: Defines a value (float) associated with each free parameter
2480        :param dict varRefflag: Defines a refinement flag (bool)
2481          associated with each free parameter
2482
2483        :returns: the expression as a str
2484        '''
2485        for v in self.freeVars:
2486            varSelect[v] = 0
2487            varName[v] = self.freeVars[v][0]
2488            varValue[v] = self.freeVars[v][1]
2489            varRefflag[v] = self.freeVars[v][2]
2490        for v in self.assgnVars:
2491            varSelect[v] = 1
2492            varName[v] = self.assgnVars[v]
2493        return self.expression
2494
2495    def GetVaried(self):
2496        'Returns the names of the free parameters that will be refined'
2497        return ["::"+self.freeVars[v][0] for v in self.freeVars if self.freeVars[v][2]]
2498
2499    def GetVariedVarVal(self):
2500        'Returns the names and values of the free parameters that will be refined'
2501        return [("::"+self.freeVars[v][0],self.freeVars[v][1]) for v in self.freeVars if self.freeVars[v][2]]
2502
2503    def UpdateVariedVars(self,varyList,values):
2504        'Updates values for the free parameters (after a refinement); only updates refined vars'
2505        for v in self.freeVars:
2506            if not self.freeVars[v][2]: continue
2507            if "::"+self.freeVars[v][0] not in varyList: continue
2508            indx = list(varyList).index("::"+self.freeVars[v][0])
2509            self.freeVars[v][1] = values[indx]
2510
2511    def GetIndependentVars(self):
2512        'Returns the names of the required independent parameters used in expression'
2513        return [self.assgnVars[v] for v in self.assgnVars]
2514
2515    def CheckVars(self):
2516        '''Check that the expression can be parsed, all functions are
2517        defined and that input loaded into the object is internally
2518        consistent. If not an Exception is raised.
2519
2520        :returns: a dict with references to packages needed to
2521          find functions referenced in the expression.
2522        '''
2523        ret = self.ParseExpression(self.expression)
2524        if not ret:
2525            raise Exception("Expression parse error")
2526        exprLblList,fxnpkgdict = ret
2527        # check each var used in expression is defined
2528        defined = list(self.assgnVars.keys()) + list(self.freeVars.keys())
2529        notfound = []
2530        for var in exprLblList:
2531            if var not in defined:
2532                notfound.append(var)
2533        if notfound:
2534            msg = 'Not all variables defined'
2535            msg1 = 'The following variables were not defined: '
2536            msg2 = ''
2537            for var in notfound:
2538                if msg: msg += ', '
2539                msg += var
2540            self.lastError = (msg1,'  '+msg2)
2541            raise Exception(msg)
2542        return fxnpkgdict
2543
2544    def ParseExpression(self,expr):
2545        '''Parse an expression and return a dict of called functions and
2546        the variables used in the expression. Returns None in case an error
2547        is encountered. If packages are referenced in functions, they are loaded
2548        and the functions are looked up into the modules global
2549        workspace.
2550
2551        Note that no changes are made to the object other than
2552        saving an error message, so that this can be used for testing prior
2553        to the save.
2554
2555        :returns: a list of used variables
2556        '''
2557        self.lastError = ('','')
2558        import ast
2559        def FindFunction(f):
2560            '''Find the object corresponding to function f
2561            :param str f: a function name such as 'numpy.exp'
2562            :returns: (pkgdict,pkgobj) where pkgdict contains a dict
2563              that defines the package location(s) and where pkgobj
2564              defines the object associated with the function.
2565              If the function is not found, pkgobj is None.
2566            '''
2567            df = f.split('.')
2568            pkgdict = {}
2569            # no listed package, try in current namespace
2570            if len(df) == 1:
2571                try:
2572                    fxnobj = eval(f)
2573                    return pkgdict,fxnobj
2574                except (AttributeError, NameError):
2575                    return None,None
2576            else:
2577                try:
2578                    fxnobj = eval(f)
2579                    pkgdict[df[0]] = eval(df[0])
2580                    return pkgdict,fxnobj
2581                except (AttributeError, NameError):
2582                    pass
2583            # includes a package, lets try to load the packages
2584            pkgname = ''
2585            path = sys.path+['./',]
2586            for pkg in f.split('.')[:-1]: # if needed, descend down the tree
2587                if pkgname:
2588                    pkgname += '.' + pkg
2589                else:
2590                    pkgname = pkg
2591                fp = None
2592                try:
2593                    fp, fppath,desc = imp.find_module(pkg,path)
2594                    pkgobj = imp.load_module(pkg,fp,fppath,desc)
2595                    pkgdict[pkgname] = pkgobj
2596                    path = [fppath]
2597                except Exception as msg:
2598                    print('load of '+pkgname+' failed with error='+str(msg))
2599                    return {},None
2600                finally:
2601                    if fp: fp.close()
2602                try:
2603                    #print 'before',pkgdict.keys()
2604                    fxnobj = eval(f,globals(),pkgdict)
2605                    #print 'after 1',pkgdict.keys()
2606                    #fxnobj = eval(f,pkgdict)
2607                    #print 'after 2',pkgdict.keys()
2608                    return pkgdict,fxnobj
2609                except:
2610                    continue
2611            return None # not found
2612        def ASTtransverse(node,fxn=False):
2613            '''Transverse a AST-parsed expresson, compiling a list of variables
2614            referenced in the expression. This routine is used recursively.
2615
2616            :returns: varlist,fxnlist where
2617              varlist is a list of referenced variable names and
2618              fxnlist is a list of used functions
2619            '''
2620            varlist = []
2621            fxnlist = []
2622            if isinstance(node, list):
2623                for b in node:
2624                    v,f = ASTtransverse(b,fxn)
2625                    varlist += v
2626                    fxnlist += f
2627            elif isinstance(node, ast.AST):
2628                for a, b in ast.iter_fields(node):
2629                    if isinstance(b, ast.AST):
2630                        if a == 'func':
2631                            fxnlist += ['.'.join(ASTtransverse(b,True)[0])]
2632                            continue
2633                        v,f = ASTtransverse(b,fxn)
2634                        varlist += v
2635                        fxnlist += f
2636                    elif isinstance(b, list):
2637                        v,f = ASTtransverse(b,fxn)
2638                        varlist += v
2639                        fxnlist += f
2640                    elif node.__class__.__name__ == "Name":
2641                        varlist += [b]
2642                    elif fxn and node.__class__.__name__ == "Attribute":
2643                        varlist += [b]
2644            return varlist,fxnlist
2645        try:
2646            exprast = ast.parse(expr)
2647        except SyntaxError:
2648            s = ''
2649            import traceback
2650            for i in traceback.format_exc().splitlines()[-3:-1]:
2651                if s: s += "\n"
2652                s += str(i)
2653            self.lastError = ("Error parsing expression:",s)
2654            return
2655        # find the variables & functions
2656        v,f = ASTtransverse(exprast)
2657        varlist = sorted(list(set(v)))
2658        fxnlist = list(set(f))
2659        pkgdict = {}
2660        # check the functions are defined
2661        for fxn in fxnlist:
2662            fxndict,fxnobj = FindFunction(fxn)
2663            if not fxnobj:
2664                self.lastError = ("Error: Invalid function",fxn,
2665                                  "is not defined")
2666                return
2667            if not hasattr(fxnobj,'__call__'):
2668                self.lastError = ("Error: Not a function.",fxn,
2669                                  "cannot be called as a function")
2670                return
2671            pkgdict.update(fxndict)
2672        return varlist,pkgdict
2673
2674    def GetDepVar(self):
2675        'return the dependent variable, or None'
2676        return self.depVar
2677
2678    def SetDepVar(self,var):
2679        'Set the dependent variable, if used'
2680        self.depVar = var
2681#==========================================================================
2682class ExpressionCalcObj(object):
2683    '''An object used to evaluate an expression from a :class:`ExpressionObj`
2684    object.
2685
2686    :param ExpressionObj exprObj: a :class:`~ExpressionObj` expression object with
2687      an expression string and mappings for the parameter labels in that object.
2688    '''
2689    def __init__(self,exprObj):
2690        self.eObj = exprObj
2691        'The expression and mappings; a :class:`ExpressionObj` object'
2692        self.compiledExpr = None
2693        'The expression as compiled byte-code'
2694        self.exprDict = {}
2695        '''dict that defines values for labels used in expression and packages
2696        referenced by functions
2697        '''
2698        self.lblLookup = {}
2699        '''Lookup table that specifies the expression label name that is
2700        tied to a particular GSAS-II parameters in the parmDict.
2701        '''
2702        self.fxnpkgdict = {}
2703        '''a dict with references to packages needed to
2704        find functions referenced in the expression.
2705        '''
2706        self.varLookup = {}
2707        '''Lookup table that specifies the GSAS-II variable(s)
2708        indexed by the expression label name. (Used for only for diagnostics
2709        not evaluation of expression.)
2710        '''
2711        self.su = None
2712        '''Standard error evaluation where supplied by the evaluator
2713        '''
2714        # Patch: for old-style expressions with a (now removed step size)
2715        if '2' in platform.python_version_tuple()[0]: 
2716            basestr = basestring
2717        else:
2718            basestr = str
2719        for v in self.eObj.assgnVars:
2720            if not isinstance(self.eObj.assgnVars[v], basestr):
2721                self.eObj.assgnVars[v] = self.eObj.assgnVars[v][0]
2722        self.parmDict = {}
2723        '''A copy of the parameter dictionary, for distance and angle computation
2724        '''
2725
2726    def SetupCalc(self,parmDict):
2727        '''Do all preparations to use the expression for computation.
2728        Adds the free parameter values to the parameter dict (parmDict).
2729        '''
2730        if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
2731            return
2732        self.fxnpkgdict = self.eObj.CheckVars()
2733        # all is OK, compile the expression
2734        self.compiledExpr = compile(self.eObj.expression,'','eval')
2735
2736        # look at first value in parmDict to determine its type
2737        parmsInList = True
2738        if '2' in platform.python_version_tuple()[0]: 
2739            basestr = basestring
2740        else:
2741            basestr = str
2742        for key in parmDict:
2743            val = parmDict[key]
2744            if isinstance(val, basestr):
2745                parmsInList = False
2746                break
2747            try: # check if values are in lists
2748                val = parmDict[key][0]
2749            except (TypeError,IndexError):
2750                parmsInList = False
2751            break
2752
2753        # set up the dicts needed to speed computations
2754        self.exprDict = {}
2755        self.lblLookup = {}
2756        self.varLookup = {}
2757        for v in self.eObj.freeVars:
2758            varname = self.eObj.freeVars[v][0]
2759            varname = "::" + varname.lstrip(':').replace(' ','_').replace(':',';')
2760            self.lblLookup[varname] = v
2761            self.varLookup[v] = varname
2762            if parmsInList:
2763                parmDict[varname] = [self.eObj.freeVars[v][1],self.eObj.freeVars[v][2]]
2764            else:
2765                parmDict[varname] = self.eObj.freeVars[v][1]
2766            self.exprDict[v] = self.eObj.freeVars[v][1]
2767        for v in self.eObj.assgnVars:
2768            varname = self.eObj.assgnVars[v]
2769            if varname in parmDict:
2770                self.lblLookup[varname] = v
2771                self.varLookup[v] = varname
2772                if parmsInList:
2773                    self.exprDict[v] = parmDict[varname][0]
2774                else:
2775                    self.exprDict[v] = parmDict[varname]
2776            elif '*' in varname:
2777                varlist = LookupWildCard(varname,list(parmDict.keys()))
2778                if len(varlist) == 0:
2779                    raise Exception("No variables match "+str(v))
2780                for var in varlist:
2781                    self.lblLookup[var] = v
2782                if parmsInList:
2783                    self.exprDict[v] = np.array([parmDict[var][0] for var in varlist])
2784                else:
2785                    self.exprDict[v] = np.array([parmDict[var] for var in varlist])
2786                self.varLookup[v] = [var for var in varlist]
2787            else:
2788                self.exprDict[v] = None
2789#                raise Exception,"No value for variable "+str(v)
2790        self.exprDict.update(self.fxnpkgdict)
2791
2792    def UpdateVars(self,varList,valList):
2793        '''Update the dict for the expression with a set of values
2794        :param list varList: a list of variable names
2795        :param list valList: a list of corresponding values
2796        '''
2797        for var,val in zip(varList,valList):
2798            self.exprDict[self.lblLookup.get(var,'undefined: '+var)] = val
2799
2800    def UpdateDict(self,parmDict):
2801        '''Update the dict for the expression with values in a dict
2802        :param dict parmDict: a dict of values, items not in use are ignored
2803        '''
2804        if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
2805            self.parmDict = parmDict
2806            return
2807        for var in parmDict:
2808            if var in self.lblLookup:
2809                self.exprDict[self.lblLookup[var]] = parmDict[var]
2810
2811    def EvalExpression(self):
2812        '''Evaluate an expression. Note that the expression
2813        and mapping are taken from the :class:`ExpressionObj` expression object
2814        and the parameter values were specified in :meth:`SetupCalc`.
2815        :returns: a single value for the expression. If parameter
2816        values are arrays (for example, from wild-carded variable names),
2817        the sum of the resulting expression is returned.
2818
2819        For example, if the expression is ``'A*B'``,
2820        where A is 2.0 and B maps to ``'1::Afrac:*'``, which evaluates to::
2821
2822        [0.5, 1, 0.5]
2823
2824        then the result will be ``4.0``.
2825        '''
2826        self.su = None
2827        if self.eObj.expression.startswith('Dist'):
2828#            GSASIIpath.IPyBreak()
2829            dist = G2mth.CalcDist(self.eObj.distance_dict, self.eObj.distance_atoms, self.parmDict)
2830            return dist
2831        elif self.eObj.expression.startswith('Angle'):
2832            angle = G2mth.CalcAngle(self.eObj.angle_dict, self.eObj.angle_atoms, self.parmDict)
2833            return angle
2834        if self.compiledExpr is None:
2835            raise Exception("EvalExpression called before SetupCalc")
2836        try:
2837            val = eval(self.compiledExpr,globals(),self.exprDict)
2838        except TypeError:
2839            val = None
2840        if not np.isscalar(val):
2841            val = np.sum(val)
2842        return val
2843
2844class G2Exception(Exception):
2845    'A generic GSAS-II exception class'
2846    def __init__(self,msg):
2847        self.msg = msg
2848    def __str__(self):
2849        return repr(self.msg)
2850
2851class G2RefineCancel(Exception):
2852    'Raised when Cancel is pressed in a refinement dialog'
2853    def __init__(self,msg):
2854        self.msg = msg
2855    def __str__(self):
2856        return repr(self.msg)
2857   
2858def HowDidIgetHere(wherecalledonly=False):
2859    '''Show a traceback with calls that brought us to the current location.
2860    Used for debugging.
2861    '''
2862    import traceback
2863    if wherecalledonly:
2864        i = traceback.format_list(traceback.extract_stack()[:-1])[-2]
2865        print(i.strip().rstrip())
2866    else:
2867        print (70*'*')
2868        for i in traceback.format_list(traceback.extract_stack()[:-1]): print(i.strip().rstrip())
2869        print (70*'*')
2870
2871# Note that this is GUI code and should be moved at somepoint
2872def CreatePDFitems(G2frame,PWDRtree,ElList,Qlimits,numAtm=1,FltBkg=0,PDFnames=[]):
2873    '''Create and initialize a new set of PDF tree entries
2874
2875    :param Frame G2frame: main GSAS-II tree frame object
2876    :param str PWDRtree: name of PWDR to be used to create PDF item
2877    :param dict ElList: data structure with composition
2878    :param list Qlimits: Q limits to be used for computing the PDF
2879    :param float numAtm: no. atom in chemical formula
2880    :param float FltBkg: flat background value
2881    :param list PDFnames: previously used PDF names
2882
2883    :returns: the Id of the newly created PDF entry
2884    '''
2885    PDFname = 'PDF '+PWDRtree[4:] # this places two spaces after PDF, which is needed is some places
2886    if PDFname in PDFnames:
2887        print('Skipping, entry already exists: '+PDFname)
2888        return None
2889    #PDFname = MakeUniqueLabel(PDFname,PDFnames)
2890    Id = G2frame.GPXtree.AppendItem(parent=G2frame.root,text=PDFname)
2891    Data = {
2892        'Sample':{'Name':PWDRtree,'Mult':1.0},
2893        'Sample Bkg.':{'Name':'','Mult':-1.0,'Refine':False},
2894        'Container':{'Name':'','Mult':-1.0,'Refine':False},
2895        'Container Bkg.':{'Name':'','Mult':-1.0},'ElList':ElList,
2896        'Geometry':'Cylinder','Diam':1.0,'Pack':0.50,'Form Vol':10.0*numAtm,'Flat Bkg':FltBkg,
2897        'DetType':'Area detector','ObliqCoeff':0.2,'Ruland':0.025,'QScaleLim':Qlimits,
2898        'Lorch':False,'BackRatio':0.0,'Rmax':100.,'noRing':False,'IofQmin':1.0,'Rmin':1.0,
2899        'I(Q)':[],'S(Q)':[],'F(Q)':[],'G(R)':[]}
2900    G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Controls'),Data)
2901    G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Peaks'),
2902        {'Limits':[1.,5.],'Background':[2,[0.,-0.2*np.pi],False],'Peaks':[]})
2903    return Id
2904#%%
2905class ShowTiming(object):
2906    '''An object to use for timing repeated sections of code.
2907
2908    Create the object with::
2909       tim0 = ShowTiming()
2910
2911    Tag sections of code to be timed with::
2912       tim0.start('start')
2913       tim0.start('in section 1')
2914       tim0.start('in section 2')
2915       
2916    etc. (Note that each section should have a unique label.)
2917
2918    After the last section, end timing with::
2919       tim0.end()
2920
2921    Show timing results with::
2922       tim0.show()
2923       
2924    '''
2925    def __init__(self):
2926        self.timeSum =  []
2927        self.timeStart = []
2928        self.label = []
2929        self.prev = None
2930    def start(self,label):
2931        import time
2932        if label in self.label:
2933            i = self.label.index(label)
2934            self.timeStart[i] = time.time()
2935        else:
2936            i = len(self.label)
2937            self.timeSum.append(0.0)
2938            self.timeStart.append(time.time())
2939            self.label.append(label)
2940        if self.prev is not None:
2941            self.timeSum[self.prev] += self.timeStart[i] - self.timeStart[self.prev]
2942        self.prev = i
2943    def end(self):
2944        import time
2945        if self.prev is not None:
2946            self.timeSum[self.prev] += time.time() - self.timeStart[self.prev]
2947        self.prev = None
2948    def show(self):
2949        sumT = sum(self.timeSum)
2950        print('Timing results (total={:.2f} sec)'.format(sumT))
2951        for i,(lbl,val) in enumerate(zip(self.label,self.timeSum)):
2952            print('{} {:20} {:8.2f} ms {:5.2f}%'.format(i,lbl,1000.*val,100*val/sumT))
2953#%%
2954
2955if __name__ == "__main__":
2956    # test variable descriptions
2957    for var in '0::Afrac:*',':1:Scale','1::dAx:0','::undefined':
2958        v = var.split(':')[2]
2959        print(var+':\t', getDescr(v),getVarStep(v))
2960    import sys; sys.exit()
2961    # test equation evaluation
2962    def showEQ(calcobj):
2963        print (50*'=')
2964        print (calcobj.eObj.expression+'='+calcobj.EvalExpression())
2965        for v in sorted(calcobj.varLookup):
2966            print ("  "+v+'='+calcobj.exprDict[v]+'='+calcobj.varLookup[v])
2967        # print '  Derivatives'
2968        # for v in calcobj.derivStep.keys():
2969        #     print '    d(Expr)/d('+v+') =',calcobj.EvalDeriv(v)
2970
2971    obj = ExpressionObj()
2972
2973    obj.expression = "A*np.exp(B)"
2974    obj.assgnVars =  {'B': '0::Afrac:1'}
2975    obj.freeVars =  {'A': [u'A', 0.5, True]}
2976    #obj.CheckVars()
2977    calcobj = ExpressionCalcObj(obj)
2978
2979    obj1 = ExpressionObj()
2980    obj1.expression = "A*np.exp(B)"
2981    obj1.assgnVars =  {'B': '0::Afrac:*'}
2982    obj1.freeVars =  {'A': [u'Free Prm A', 0.5, True]}
2983    #obj.CheckVars()
2984    calcobj1 = ExpressionCalcObj(obj1)
2985
2986    obj2 = ExpressionObj()
2987    obj2.distance_stuff = np.array([[0,1],[1,-1]])
2988    obj2.expression = "Dist(1,2)"
2989    GSASIIpath.InvokeDebugOpts()
2990    parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
2991    calcobj2 = ExpressionCalcObj(obj2)
2992    calcobj2.SetupCalc(parmDict2)
2993    showEQ(calcobj2)
2994
2995    parmDict1 = {'0::Afrac:0':1.0, '0::Afrac:1': 1.0}
2996    print ('\nDict = '+parmDict1)
2997    calcobj.SetupCalc(parmDict1)
2998    showEQ(calcobj)
2999    calcobj1.SetupCalc(parmDict1)
3000    showEQ(calcobj1)
3001
3002    parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
3003    print ('Dict = '+parmDict2)
3004    calcobj.SetupCalc(parmDict2)
3005    showEQ(calcobj)
3006    calcobj1.SetupCalc(parmDict2)
3007    showEQ(calcobj1)
3008    calcobj2.SetupCalc(parmDict2)
3009    showEQ(calcobj2)
Note: See TracBrowser for help on using the repository browser.