Context Navigation

source: trunk/GSASIIobj.py @ 5360

Last change on this file since 5360 was 5307, checked in by vondreele, 3 years ago
add micro-ED support. New electron scattering tables, new histogram type ('SEC'), etc., etc. Fix sign of ZERO for RMCProfile MakeInst? fxn. Add calc of Rvalues to Error Analysis for PWDR & HKLF histograms; results are printed om console. Add a stub for Cluster analysis in Controls GUI
Property svn:eol-style set to `native` Property svn:keywords set to `Date Author Revision URL Id`
File size: 172.3 KB

Line
1	# -- coding: utf-8 --
2	#GSASIIobj - data objects for GSAS-II
3	########### SVN repository information ###################
4	# $Date: 2022-07-07 17:02:04 +0000 (Thu, 07 Jul 2022) $
5	# $Author: toby $
6	# $Revision: 5307 $
7	# $URL: trunk/GSASIIobj.py $
8	# $Id: GSASIIobj.py 5307 2022-07-07 17:02:04Z toby $
9	########### SVN repository information ###################
10
11	'''
12	GSASIIobj: Data objects
13	=========================
14
15	This module defines and/or documents the data structures used in GSAS-II, as well
16	as provides misc. support routines.
17
18	.. Next command allows \\AA to be used in HTML
19
20	.. only:: html
21
22	:math:`\\require{mediawiki-texvc}`
23
24
25	.. _VarNames_table:
26
27	.. index::
28	single: Parameter names
29	single: GSAS-II variable naming
30
31	Variable names in GSAS-II
32	----------------------------
33
34	Parameter are named using the following pattern,
35	``p:h:<var>:n``, where ``<var>`` is a variable name, as shown in the following table. Also,
36	``p`` is the phase number, ``h`` is the histogram number,
37	and ``n`` is the atom parameter number
38	If a parameter does not depend on a histogram, phase or atom, ``h``, ``p`` and/or ``n`` will be omitted,
39	so ``p::<var>:n``, ``:h:<var>`` and ``p:h:<var>`` are all valid names.
40
41	.. include:: vars.rst
42
43	.. _Constraints_table:
44
45	.. index::
46	single: Constraints object description
47	single: Data object descriptions; Constraints
48
49	Constraints Tree Item
50	----------------------
51
52	Constraints are stored in a dict, separated into groups.
53	Note that parameter are named in the following pattern,
54	p:h:<var>:n, where p is the phase number, h is the histogram number
55	<var> is a variable name and n is the parameter number.
56	If a parameter does not depend on a histogram or phase or is unnumbered, that
57	number is omitted.
58	Note that the contents of each dict item is a List where each element in the
59	list is a :ref:`constraint definition objects <Constraint_definitions_table>`.
60	The constraints in this form are converted in
61	:func:`GSASIImapvars.ProcessConstraints` to the form used in :mod:`GSASIImapvars`
62
63	The keys in the Constraints dict are:
64
65	.. tabularcolumns:: \|l\|p{4.5in}\|
66
67	========== ====================================================
68	key explanation
69	========== ====================================================
70	Hist This specifies a list of constraints on
71	histogram-related parameters,
72	which will be of form :h:<var>:n.
73	HAP This specifies a list of constraints on parameters
74	that are defined for every histogram in each phase
75	and are of form p:h:<var>:n.
76	Phase This specifies a list of constraints on phase
77	parameters,
78	which will be of form p::<var>:n.
79	Global This specifies a list of constraints on parameters
80	that are not tied to a histogram or phase and
81	are of form ::<var>:n
82	========== ====================================================
83
84	.. _Constraint_definitions_table:
85
86	.. index::
87	single: Constraint definition object description
88	single: Data object descriptions; Constraint Definition
89
90	Each constraint is defined as an item in a list. Each constraint is of form::
91
92	[[<mult1>, <var1>], [<mult2>, <var2>],..., <fixedval>, <varyflag>, <constype>]
93
94	Where the variable pair list item containing two values [<mult>, <var>], where:
95
96	* <mult> is a multiplier for the constraint (float)
97	* <var> a :class:`G2VarObj` object. (Note that in very old .gpx files this might be a str with a variable name of form 'p:h:name[:at]')
98
99	Note that the last three items in the list play a special role:
100
101	* <fixedval> is the fixed value for a `constant equation` (``constype=c``)
102	constraint or is None. For a `New variable` (``constype=f``) constraint,
103	a variable name can be specified as a str (used for externally
104	generated constraints)
105	* <varyflag> is True or False for `New variable` (``constype=f``) constraints
106	or is None. This indicates if this variable should be refined.
107	* <constype> is one of four letters, 'e', 'c', 'h', 'f' that determines the type of constraint:
108
109	* 'e' defines a set of equivalent variables. Only the first variable is refined (if the
110	appropriate refine flag is set) and and all other equivalent variables in the list
111	are generated from that variable, using the appropriate multipliers.
112	* 'c' defines a constraint equation of form,
113	:math:`m_1 \\times var_1 + m_2 \\times var_2 + ... = c`
114	* 'h' defines a variable to hold (not vary). Any variable on this list is not varied,
115	even if its refinement flag is set. Only one [mult,var] pair is allowed in a hold
116	constraint and the mult value is ignored.
117	This is of particular value when needing to hold one or more variables where a
118	single flag controls a set of variables such as, coordinates,
119	the reciprocal metric tensor or anisotropic displacement parameter.
120	* 'f' defines a new variable (function) according to relationship
121	:math:`newvar = m_1 \\times var_1 + m_2 \\times var_2 + ...`
122
123	.. _Covariance_table:
124
125	.. index::
126	single: Covariance description
127	single: Data object descriptions; Covariance
128
129	Covariance Tree Item
130	--------------------
131
132	The Covariance tree item has results from the last least-squares run. They
133	are stored in a dict with these keys:
134
135	.. tabularcolumns:: \|l\|l\|p{4in}\|
136
137	============= =============== ====================================================
138	key sub-key explanation
139	============= =============== ====================================================
140	newCellDict \\ (dict) ith lattice parameters computed by
141	:func:`GSASIIstrMath.GetNewCellParms`
142	title \\ (str) Name of gpx file(?)
143	variables \\ (list) Values for all N refined variables
144	(list of float values, length N,
145	ordered to match varyList)
146	sig \\ (list) Uncertainty values for all N refined variables
147	(list of float values, length N,
148	ordered to match varyList)
149	varyList \\ (list of str values, length N) List of directly refined variables
150
151	newAtomDict \\ (dict) atom position values computed in
152	:func:`GSASIIstrMath.ApplyXYZshifts`
153	Rvals \\ (dict) R-factors, GOF, Marquardt value for last
154	refinement cycle
155	\\ Nobs (int) Number of observed data points
156	\\ Rwp (float) overall weighted profile R-factor (%)
157	\\ chisq (float) :math:`\\sum w*(I_{obs}-I_{calc})^2`
158	for all data.
159	Note: this is not the reduced :math:`\\chi^2`.
160	\\ lamMax (float) Marquardt value applied to Hessian diagonal
161	\\ GOF (float) The goodness-of-fit, aka square root of
162	the reduced chi squared.
163	covMatrix \\ (np.array) The (NxN) covVariance matrix
164	============= =============== ====================================================
165
166	.. _Phase_table:
167
168	.. index::
169	single: Phase object description
170	single: Data object descriptions; Phase
171
172	Phase Tree Items
173	----------------
174
175	Phase information is stored in the GSAS-II data tree as children of the
176	Phases item in a dict with keys:
177
178	.. tabularcolumns:: \|l\|l\|p{4in}\|
179
180	========== =============== =====================================================================================================
181	key sub-key explanation
182	========== =============== =====================================================================================================
183	General \\ (dict) Overall information for the phase
184	\\ 3Dproj (list of str) projections for 3D pole distribution plots
185	\\ AngleRadii (list of floats) Default radius for each atom used to compute
186	interatomic angles
187	\\ AtomMass (list of floats) Masses for atoms
188	\\ AtomPtrs (list of int) four locations (cx,ct,cs & cu) to use to pull info
189	from the atom records
190	\\ AtomTypes (llist of str) Atom types
191	\\ BondRadii (list of floats) Default radius for each atom used to compute
192	interatomic distances
193	\\ Cell Unit cell parameters & ref. flag
194	(list with 8 items. All but first item are float.)
195
196	0: cell refinement flag (True/False),
197
198	1-3: a, b, c, (:math:`\\AA`)
199
200	4-6: alpha, beta & gamma, (degrees)
201
202	7: volume (:math:`\\AA^3`)
203
204	\\ Color (list of (r,b,g) triplets) Colors for atoms
205	\\ Compare (dict) Polygon comparison parameters
206	\\ Data plot type (str) data plot type ('Mustrain', 'Size' or
207	'Preferred orientation') for powder data
208	\\ DisAglCtls (dDict) with distance/angle search controls,
209	which has keys 'Name', 'AtomTypes',
210	'BondRadii', 'AngleRadii' which are as above
211	except are possibly edited. Also contains
212	'Factors', which is a 2 element list with
213	a multiplier for bond and angle search range
214	[typically (0.85,0.85)].
215	\\ F000X (float) x-ray F(000) intensity
216	\\ F000N (float) neutron F(000) intensity
217	\\ Flip (dict) Charge flip controls
218	\\ HydIds (dict) geometrically generated hydrogen atoms
219	\\ Isotope (dict) Isotopes for each atom type
220	\\ Isotopes (dict) Scattering lengths for each isotope
221	combination for each element in phase
222	\\ MCSA controls (dict) Monte Carlo-Simulated Annealing controls
223	\\ Map (dict) Map parameters
224	\\ Mass (float) Mass of unit cell contents in g/mol
225	\\ Modulated (bool) True if phase modulated
226	\\ Mydir (str) Directory of current .gpx file
227	\\ Name (str) Phase name
228	\\ NoAtoms (dict) Number of atoms per unit cell of each type
229	\\ POhkl (list) March-Dollase preferred orientation direction
230	\\ Pawley dmin (float) maximum Q (as d-space) to use for Pawley extraction
231	\\ Pawley dmax (float) minimum Q (as d-space) to use for Pawley extraction
232	\\ Pawley neg wt (float) Restraint value for negative Pawley intensities
233	\\ SGData (object) Space group details as a
234	:ref:`space group (SGData) <SGData_table>`
235	object, as defined in :func:`GSASIIspc.SpcGroup`.
236	\\ SH Texture (dict) Spherical harmonic preferred orientation parameters
237	\\ Super (int) dimension of super group (0,1 only)
238	\\ Type (str) phase type (e.g. 'nuclear')
239	\\ Z (dict) Atomic numbers for each atom type
240	\\ doDysnomia (bool) flag for max ent map modification via Dysnomia
241	\\ doPawley (bool) Flag for Pawley intensity extraction
242	\\ vdWRadii (dict) Van der Waals radii for each atom type
243	ranId \\ (int) unique random number Id for phase
244	pId \\ (int) Phase Id number for current project.
245	Atoms \\ (list of lists) Atoms in phase as a list of lists. The outer list
246	is for each atom, the inner list contains varying
247	items depending on the type of phase, see
248	the :ref:`Atom Records <Atoms_table>` description.
249	Drawing \\ (dict) Display parameters
250	\\ Atoms (list of lists) with an entry for each atom that is drawn
251	\\ Plane (list) Controls for contour density plane display
252	\\ Quaternion (4 element np.array) Viewing quaternion
253	\\ Zclip (float) clipping distance in :math:`\\AA`
254	\\ Zstep (float) Step to de/increase Z-clip
255	\\ atomPtrs (list) positions of x, type, site sym, ADP flag in Draw Atoms
256	\\ backColor (list) background for plot as and R,G,B triplet
257	(default = [0, 0, 0], black).
258	\\ ballScale (float) Radius of spheres in ball-and-stick display
259	\\ bondList (dict) Bonds
260	\\ bondRadius (float) Radius of binds in :math:`\\AA`
261	\\ cameraPos (float) Viewing position in :math:`\\AA` for plot
262	\\ contourLevel (float) map contour level in :math:`e/\\AA^3`
263	\\ contourMax (float) map contour maximum
264	\\ depthFog (bool) True if use depthFog on plot - set currently as False
265	\\ ellipseProb (float) Probability limit for display of thermal
266	ellipsoids in % .
267	\\ magMult (float) multiplier for magnetic moment arrows
268	\\ mapSize (float) x & y dimensions of contourmap (fixed internally)
269	\\ modelView (4,4 array) from openGL drawing transofmation matrix
270	\\ oldxy (list with two floats) previous view point
271	\\ radiusFactor (float) Distance ratio for searching for bonds. Bonds
272	are located that are within r(Ra+Rb) and (Ra+Rb)/r
273	where Ra and Rb are the atomic radii.
274	\\ selectedAtoms (list of int values) List of selected atoms
275	\\ showABC (bool) Flag to show view point triplet. True=show.
276	\\ showHydrogen (bool) Flag to control plotting of H atoms.
277	\\ showRigidBodies (bool) Flag to highlight rigid body placement
278	\\ showSlice (bool) flag to show contour map
279	\\ sizeH (float) Size ratio for H atoms
280	\\ unitCellBox (bool) Flag to control display of the unit cell.
281	\\ vdwScale (float) Multiplier of van der Waals radius for display of vdW spheres.
282	\\ viewDir (np.array with three floats) cartesian viewing direction
283	\\ viewPoint (list of lists) First item in list is [x,y,z]
284	in fractional coordinates for the center of
285	the plot. Second item list of previous & current
286	atom number viewed (may be [0,0])
287	ISODISTORT \\ (dict) contains controls for running ISODISTORT and results from it
288	\\ ISOmethod (int) ISODISTORT method (currently 1 or 4; 2 & 3 not implemented in GSAS-II)
289	\\ ParentCIF (str) parent cif file name for ISODISTORT method 4
290	\\ ChildCIF (str) child cif file name for ISODISTORT method 4
291	\\ SGselect (dict) selection list for lattice types in radio result from ISODISTORT method 1
292	\\ selection (int) chosen selection from radio
293	\\ radio (list) results from ISODISTORT method 1
294	\\ ChildMatrix (3x3 array) transformation matrix for method 3 (not currently used)
295	\\ ChildSprGp (str) child space group for method 3 (not currently used)
296	\\ ChildCell (str) cell ordering for nonstandard orthorhombic ChildSprGrp in method 3 (not currently used)
297	\\ G2ModeList (list) ISODISTORT mode names
298	\\ modeDispl (list) distortion mode values; refinable parameters
299	\\ ISOmodeDispl (list) distortion mode values as determined in method 4 by ISODISTORT
300	\\ NormList (list) ISODISTORT normalization values; to convert mode value to fractional coordinate dsplacement
301	\\ G2parentCoords (list) full set of parent structure coordinates transformed to child structure; starting basis for mode displacements
302	\\ G2VarList (list)
303	\\ IsoVarList (list)
304	\\ G2coordOffset (list) only adjustible set of parent structure coordinates
305	\\ G2OccVarList (list)
306	\\ Var2ModeMatrix (array) atom variable to distortion mode transformation
307	\\ Mode2VarMatrix (array) distortion mode to atom variable transformation
308	\\ rundata (dict) saved input information for use by ISODISTORT method 1
309
310	RBModels \\ Rigid body assignments (note Rigid body definitions
311	are stored in their own main top-level tree entry.)
312	RMC \\ (dict) RMCProfile, PDFfit & fullrmc controls
313	Pawley ref \\ (list) Pawley reflections
314	Histograms \\ (dict of dicts) The key for the outer dict is
315	the histograms tied to this phase. The inner
316	dict contains the combined phase/histogram
317	parameters for items such as scale factors,
318	size and strain parameters. The following are the
319	keys to the inner dict. (dict)
320	\\ Babinet (dict) For protein crystallography. Dictionary with two
321	entries, 'BabA', 'BabU'
322	\\ Extinction (list of float, bool) Extinction parameter
323	\\ Flack (list of [float, bool]) Flack parameter & refine flag
324	\\ HStrain (list of two lists) Hydrostatic strain. The first is
325	a list of the HStrain parameters (1, 2, 3, 4, or 6
326	depending on unit cell), the second is a list of boolean
327	refinement parameters (same length)
328	\\ Histogram (str) The name of the associated histogram
329	\\ Layer Disp (list of [float, bool]) Layer displacement in beam direction & refine flag
330	\\ LeBail (bool) Flag for LeBail extraction
331	\\ Mustrain (list) Microstrain parameters, in order:
332
333	0. Type, one of u'isotropic', u'uniaxial', u'generalized'
334	1. Isotropic/uniaxial parameters - list of 3 floats
335	2. Refinement flags - list of 3 bools
336	3. Microstrain axis - list of 3 ints, [h, k, l]
337	4. Generalized mustrain parameters - list of 2-6 floats, depending on space group
338	5. Generalized refinement flags - list of bools, corresponding to the parameters of (4)
339	\\ Pref.Ori. (list) Preferred Orientation. List of eight parameters.
340	Items marked SH are only used for Spherical Harmonics.
341
342	0. (str) Type, 'MD' for March-Dollase or 'SH' for Spherical Harmonics
343	1. (float) Value
344	2. (bool) Refinement flag
345	3. (list) Preferred direction, list of ints, [h, k, l]
346	4. (int) SH - number of terms
347	5. (dict) SH -
348	6. (list) SH
349	7. (float) SH
350	\\ Scale (list of [float, bool]) Phase fraction & refine flag
351	\\ Size List of crystallite size parameters, in order:
352
353	0. (str) Type, one of u'isotropic', u'uniaxial', u'ellipsoidal'
354	1. (list) Isotropic/uniaxial parameters - list of 3 floats
355	2. (list) Refinement flags - list of 3 bools
356	3. (list) Size axis - list of 3 ints, [h, k, l]
357	4. (list) Ellipsoidal size parameters - list of 6 floats
358	5. (list) Ellipsoidal refinement flags - list of bools, corresponding to the parameters of (4)
359	\\ Use (bool) True if this histogram is to be used in refinement
360	MCSA \\ (dict) Monte-Carlo simulated annealing parameters
361	========== =============== =====================================================================================================
362
363	.. _RBData_table:
364
365	.. index::
366	single: Rigid Body Data description
367	single: Data object descriptions; Rigid Body Data
368
369	Rigid Body Objects
370	------------------
371
372	Rigid body descriptions are available for two types of rigid bodies: 'Vector'
373	and 'Residue'. Vector rigid bodies are developed by a sequence of translations each
374	with a refinable magnitude and Residue rigid bodies are described as Cartesian coordinates
375	with defined refinable torsion angles.
376
377	.. tabularcolumns:: \|l\|l\|p{4in}\|
378
379	========== =============== ====================================================
380	key sub-key explanation
381	========== =============== ====================================================
382	Vector RBId (dict of dict) vector rigid bodies
383	\\ AtInfo (dict) Drad, Color: atom drawing radius & color for each atom type
384	\\ RBname (str) Name assigned by user to rigid body
385	\\ VectMag (list) vector magnitudes in :math:`\\AA`
386	\\ rbXYZ (list of 3 float Cartesian coordinates for Vector rigid body )
387	\\ rbRef (list of 3 int & 1 bool) 3 assigned reference atom nos. in rigid body for origin
388	definition, use center of atoms flag
389	\\ VectRef (list of bool refinement flags for VectMag values )
390	\\ rbTypes (list of str) Atom types for each atom in rigid body
391	\\ rbVect (list of lists) Cartesian vectors for each translation used to build rigid body
392	\\ useCount (int) Number of times rigid body is used in any structure
393	Residue RBId (dict of dict) residue rigid bodies
394	\\ AtInfo (dict) Drad, Color: atom drawing radius & color for each atom type
395	\\ RBname (str) Name assigned by user to rigid body
396	\\ rbXYZ (list of 3 float) Cartesian coordinates for Residue rigid body
397	\\ rbTypes (list of str) Atom types for each atom in rigid body
398	\\ atNames (list of str) Names of each atom in rigid body (e.g. C1,N2...)
399	\\ rbRef (list of 3 int & 1 bool) 3 assigned reference atom nos. in rigid body for origin
400	definition, use center of atoms flag
401	\\ rbSeq (list) Orig,Piv,angle,Riding : definition of internal rigid body
402	torsion; origin atom (int), pivot atom (int), torsion angle (float),
403	riding atoms (list of int)
404	\\ SelSeq (int,int) used by SeqSizer to identify objects
405	\\ useCount (int)Number of times rigid body is used in any structure
406	RBIds \\ (dict) unique Ids generated upon creation of each rigid body
407	\\ Vector (list) Ids for each Vector rigid body
408	\\ Residue (list) Ids for each Residue rigid body
409	========== =============== ====================================================
410
411	.. _SGData_table:
412
413	.. index::
414	single: Space Group Data description
415	single: Data object descriptions; Space Group Data
416
417	Space Group Objects
418	-------------------
419
420	Space groups are interpreted by :func:`GSASIIspc.SpcGroup`
421	and the information is placed in a SGdata object
422	which is a dict with these keys. Magnetic ones are marked "mag"
423
424	.. tabularcolumns:: \|l\|p{4.5in}\|
425
426	========== ========================================================================================
427	key explanation
428	========== ========================================================================================
429	BNSlattsym mag - (str) BNS magnetic space group symbol and centering vector
430	GenFlg mag - (list) symmetry generators indices
431	GenSym mag - (list) names for each generator
432	MagMom mag - (list) "time reversals" for each magnetic operator
433	MagPtGp mag - (str) Magnetic point group symbol
434	MagSpGrp mag - (str) Magnetic space group symbol
435	OprNames mag - (list) names for each space group operation
436	SGCen (np.array) Symmetry cell centering vectors. A (n,3) np.array
437	of centers. Will always have at least one row: ``np.array([[0, 0, 0]])``
438	SGFixed (bool) Only True if phase mported from a magnetic cif file
439	then the space group can not be changed by the user because
440	operator set from cif may be nonstandard
441	SGGen (list) generators
442	SGGray (bool) True if space group is a gray group (incommensurate magnetic structures)
443	SGInv (bool) True if centrosymmetric, False if not
444	SGLatt (str)Lattice centering type. Will be one of
445	P, A, B, C, I, F, R
446	SGLaue (str) one of the following 14 Laue classes:
447	-1, 2/m, mmm, 4/m, 4/mmm, 3R,
448	3mR, 3, 3m1, 31m, 6/m, 6/mmm, m3, m3m
449	SGOps (list) symmetry operations as a list of form
450	``[[M1,T1], [M2,T2],...]``
451	where :math:`M_n` is a 3x3 np.array
452	and :math:`T_n` is a length 3 np.array.
453	Atom coordinates are transformed where the
454	Asymmetric unit coordinates [X is (x,y,z)]
455	are transformed using
456	:math:`X^\\prime = M_n*X+T_n`
457	SGPolax (str) Axes for space group polarity. Will be one of
458	'', 'x', 'y', 'x y', 'z', 'x z', 'y z',
459	'xyz'. In the case where axes are arbitrary
460	'111' is used (P 1, and ?).
461	SGPtGrp (str) Point group of the space group
462	SGUniq unique axis if monoclinic. Will be
463	a, b, or c for monoclinic space groups.
464	Will be blank for non-monoclinic.
465	SGSpin mag - (list) of spin flip operatiors (+1 or -1) for the space group operations
466	SGSys (str) symmetry unit cell: type one of
467	'triclinic', 'monoclinic', 'orthorhombic',
468	'tetragonal', 'rhombohedral', 'trigonal',
469	'hexagonal', 'cubic'
470	SSGK1 (list) Superspace multipliers
471	SpGrp (str) space group symbol
472	SpnFlp mag - (list) Magnetic spin flips for every magnetic space group operator
473	========== ========================================================================================
474
475	.. _SSGData_table:
476
477	.. index::
478	single: Superspace Group Data description
479	single: Data object descriptions; Superspace Group Data
480
481	Superspace groups [3+1] are interpreted by :func:`GSASIIspc.SSpcGroup`
482	and the information is placed in a SSGdata object
483	which is a dict with these keys:
484
485	.. tabularcolumns:: \|l\|p{4.5in}\|
486
487	========== ====================================================
488	key explanation
489	========== ====================================================
490	SSGCen (list) 4D cell centering vectors [0,0,0,0] at least
491	SSGK1 (list) Superspace multipliers
492	SSGOps (list) 4D symmetry operations as [M,T] so that M*x+T = x'
493	SSpGrp (str) superspace group symbol extension to space group
494	symbol, accidental spaces removed
495	modQ (list) modulation/propagation vector
496	modSymb (list of str) Modulation symbols
497	========== ====================================================
498
499
500	Phase Information
501	--------------------
502
503	.. index::
504	single: Phase information record description
505
506	Phase information is placed in one of the following keys:
507
508	.. tabularcolumns:: \|l\|p{4.5in}\|
509
510	========== ==============================================================
511	key explanation
512	========== ==============================================================
513	General Overall information about a phase
514	Histograms Information about each histogram linked to the
515	current phase as well as parameters that
516	are defined for each histogram and phase
517	(such as sample peak widths and preferred
518	orientation parameters.
519	Atoms Contains a list of atoms, as described in the
520	:ref:`Atom Records <Atoms_table>` description.
521	Drawing Parameters that determine how the phase is
522	displayed, including a list of atoms to be
523	included, as described in the
524	:ref:`Drawing Atom Records <Drawing_atoms_table>`
525	description
526	MCSA Monte-Carlo simulated annealing parameters
527	pId The index of each phase in the project, numbered
528	starting at 0
529	ranId An int value with a unique value for each phase
530	RBModels A list of dicts with parameters for each
531	rigid body inserted into the current phase,
532	as defined in the
533	:ref:`Rigid Body Insertions <Rigid_Body_Insertions>`.
534	Note that the rigid bodies are defined as
535	:ref:`Rigid Body Objects <RBData_table>`
536	RMC PDF modeling parameters
537	Pawley ref Pawley refinement parameters
538
539	========== ==============================================================
540
541	.. _Atoms_table:
542
543	.. index::
544	single: Atoms record description
545	single: Data object descriptions; Atoms record
546
547	--------------------
548	Atom Records
549	--------------------
550
551	If ``phasedict`` points to the phase information in the data tree, then
552	atoms are contained in a list of atom records (list) in
553	``phasedict['Atoms']``. Also needed to read atom information
554	are four pointers, ``cx,ct,cs,cia = phasedict['General']['AtomPtrs']``,
555	which define locations in the atom record, as shown below. Items shown are
556	always present; additional ones for macromolecular phases are marked 'mm',
557	and those for magnetic structures are marked 'mg'
558
559	.. tabularcolumns:: \|l\|p{4.5in}\|
560
561	============== ====================================================
562	location explanation
563	============== ====================================================
564	ct-4 mm - (str) residue number
565	ct-3 mm - (str) residue name (e.g. ALA)
566	ct-2 mm - (str) chain label
567	ct-1 (str) atom label
568	ct (str) atom type
569	ct+1 (str) refinement flags; combination of 'F', 'X', 'U', 'M'
570	cx,cx+1,cx+2 (3 floats) the x,y and z coordinates
571	cx+3 (float) site occupancy
572	cx+4,cx+5,cx+6 mg - (list) atom magnetic moment along a,b,c in Bohr magnetons
573	cs (str) site symmetry
574	cs+1 (int) site multiplicity
575	cia (str) ADP flag: Isotropic ('I') or Anisotropic ('A')
576	cia+1 (float) Uiso
577	cia+2...cia+7 (6 floats) U11, U22, U33, U12, U13, U23
578	atom[cia+8] (int) unique atom identifier
579
580	============== ====================================================
581
582	.. _Drawing_atoms_table:
583
584	.. index::
585	single: Drawing atoms record description
586	single: Data object descriptions; Drawing atoms record
587
588	----------------------------
589	Drawing Atom Records
590	----------------------------
591
592	If ``phasedict`` points to the phase information in the data tree, then
593	drawing atoms are contained in a list of drawing atom records (list) in
594	``phasedict['Drawing']['Atoms']``. Also needed to read atom information
595	are four pointers, ``cx,ct,cs,ci = phasedict['Drawing']['AtomPtrs']``,
596	which define locations in the atom record, as shown below. Items shown are
597	always present; additional ones for macromolecular phases are marked 'mm',
598	and those for magnetic structures are marked 'mg'
599
600	.. tabularcolumns:: \|l\|p{4.5in}\|
601
602	============== ===================================================================================
603	location explanation
604	============== ===================================================================================
605	ct-4 mm - (str) residue number
606	ct-3 mm - (str) residue name (e.g. ALA)
607	ct-2 mm - (str) chain label
608	ct-1 (str) atom label
609	ct (str) atom type
610	cx,cx+1,cx+2 (3 floats) the x,y and z coordinates
611	cx+3,cx+4,cx+5 mg - (3 floats) atom magnetic moment along a,b,c in Bohr magnetons
612	cs-1 (str) Sym Op symbol; sym. op number + unit cell id (e.g. '1,0,-1')
613	cs (str) atom drawing style; e.g. 'balls & sticks'
614	cs+1 (str) atom label style (e.g. 'name')
615	cs+2 (int) atom color (RBG triplet)
616	cs+3 (str) ADP flag: Isotropic ('I') or Anisotropic ('A')
617	cs+4 (float) Uiso
618	cs+5...cs+11 (6 floats) U11, U22, U33, U12, U13, U23
619	ci (int) unique atom identifier; matches source atom Id in Atom Records
620	============== ===================================================================================
621
622	.. _Rigid_Body_Insertions:
623
624	----------------------------
625	Rigid Body Insertions
626	----------------------------
627
628	If ``phasedict`` points to the phase information in the data tree, then
629	rigid body information is contained in list(s) in
630	``phasedict['RBModels']['Residue']`` and/or ``phasedict['RBModels']['Vector']``
631	for each rigid body inserted into the current phase.
632
633	.. tabularcolumns:: \|l\|p{4.5in}\|
634
635	============== ===================================================================================
636	key explanation
637	============== ===================================================================================
638	fixOrig Should the origin be fixed (when editing, not the refinement flag)
639	Ids Ids for assignment of atoms in the rigid body
640	numChain Chain number for macromolecular fits
641	Orient Orientation of the RB as a quaternion and a refinement flag (' ', 'A' or 'AV')
642	OrientVec Orientation of the RB expressed as a vector and azimuthal rotation angle
643	Orig Origin of the RB in fractional coordinates and refinement flag (bool)
644	RBId References the unique ID of a rigid body in the
645	:ref:`Rigid Body Objects <RBData_table>`
646	RBname The name for the rigid body (str)
647	AtomFrac The atom fractions for the rigid body
648	ThermalMotion The thermal motion description for the rigid body, which includes a choice for
649	the model and can include TLS parameters or an overall Uiso value.
650	Torsions Defines the torsion angle and refinement flag for each torsion defined in
651	the :ref:`Rigid Body Object <RBData_table>`
652	============== ===================================================================================
653
654	.. _Powder_table:
655
656	.. index::
657	single: Powder data object description
658	single: Data object descriptions; Powder Data
659
660	Powder Diffraction Tree Items
661	-----------------------------
662
663	Every powder diffraction histogram is stored in the GSAS-II data tree
664	with a top-level entry named beginning with the string "PWDR ". The
665	diffraction data for that information are directly associated with
666	that tree item and there are a series of children to that item. The
667	routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
668	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
669	load this information into a dictionary where the child tree name is
670	used as a key, and the information in the main entry is assigned
671	a key of ``Data``, as outlined below.
672
673	.. tabularcolumns:: \|p{1in}\|p{1in}\|p{4in}\|
674
675	====================== =============== ===========================================================
676	key sub-key explanation
677	====================== =============== ===========================================================
678	Comments \\ (list of str) Text strings extracted from the original powder
679	data header. These cannot be changed by the user;
680	it may be empty.
681	Limits \\ (list) two two element lists, as [[Ld,Hd],[L,H]]
682	where L and Ld are the current and default lowest
683	two-theta value to be used and
684	where H and Hd are the current and default highest
685	two-theta value to be used.
686	Reflection Lists \\ (dict of dicts) with an entry for each phase in the
687	histogram. The contents of each dict item
688	is a dict containing reflections, as described in
689	the :ref:`Powder Reflections <PowderRefl_table>`
690	description.
691	Instrument Parameters \\ (dict) The instrument parameters uses different dicts
692	for the constant wavelength (CW) and time-of-flight (TOF)
693	cases. See below for the descriptions of each.
694	wtFactor \\ (float) A weighting factor to increase or decrease
695	the leverage of data in the histogram .
696	A value of 1.0 weights the data with their
697	standard uncertainties and a larger value
698	increases the weighting of the data (equivalent
699	to decreasing the uncertainties).
700	Sample Parameters \\ (dict) Parameters that describe how
701	the data were collected, as listed
702	below. Refinable parameters are a list containing
703	a float and a bool, where the second value
704	specifies if the value is refined, otherwise
705	the value is a float unless otherwise noted.
706	\\ Scale The histogram scale factor (refinable)
707	\\ Absorption The sample absorption coefficient as
708	:math:`\\mu r` where r is the radius
709	(refinable). Only valid for Debye-Scherrer geometry.
710	\\ SurfaceRoughA Surface roughness parameter A as defined by
711	Surotti, J. Appl. Cryst, 5, 325-331, 1972.
712	(refinable - only valid for Bragg-Brentano geometry)
713	\\ SurfaceRoughB Surface roughness parameter B (refinable -
714	only valid for Bragg-Brentano geometry)
715	\\ DisplaceX, Sample displacement from goniometer center
716	DisplaceY where Y is along the beam direction and
717	X is perpendicular. Units are :math:`\\mu m`
718	(refinable).
719	\\ Phi, Chi, Goniometer sample setting angles, in degrees.
720	Omega
721	\\ Gonio. radius Radius of the diffractometer in mm
722	\\ InstrName (str) A name for the instrument, used in preparing
723	a CIF .
724	\\ Force, Variables that describe how the measurement
725	Temperature, was performed. Not used directly in
726	Humidity, any computations.
727	Pressure,
728	Voltage
729	\\ ranId (int) The random-number Id for the histogram
730	(same value as where top-level key is ranId)
731	\\ Type (str) Type of diffraction data, may be 'Debye-Scherrer'
732	or 'Bragg-Brentano' .
733	hId \\ (int) The number assigned to the histogram when
734	the project is loaded or edited (can change)
735	ranId \\ (int) A random number id for the histogram
736	that does not change
737	Background \\ (list) The background is stored as a list with where
738	the first item in the list is list and the second
739	item is a dict. The list contains the background
740	function and its coefficients; the dict contains
741	Debye diffuse terms and background peaks.
742	(TODO: this needs to be expanded.)
743	Data \\ (list) The data consist of a list of 6 np.arrays
744	containing in order:
745
746	0. the x-postions (two-theta in degrees),
747	1. the intensity values (Yobs),
748	2. the weights for each Yobs value
749	3. the computed intensity values (Ycalc)
750	4. the background values
751	5. Yobs-Ycalc
752	====================== =============== ===========================================================
753
754	.. _CWPowder_table:
755
756	.. index::
757	single: Powder data CW Instrument Parameters
758
759	-----------------------------
760	CW Instrument Parameters
761	-----------------------------
762
763	Instrument Parameters are placed in a list of two dicts,
764	where the keys in the first dict are listed below. Note that the dict contents are different for
765	constant wavelength (CW) vs. time-of-flight (TOF) histograms.
766	The value for each item is a list containing three values: the initial value, the current value
767	and a refinement flag which can have a value of True, False or 0 where 0 indicates a value that
768	cannot be refined. The first and second values are floats unless otherwise noted.
769	Items not refined are noted as [*]
770
771	.. tabularcolumns:: \|l\|p{1in}\|p{4in}\|
772
773	======================== =============== ===========================================================
774	key sub-key explanation
775	======================== =============== ===========================================================
776	Instrument Parameters[0] Type [*] (str) Histogram type:
777	* 'PXC' for constant wavelength x-ray
778	* 'PNC' for constant wavelength neutron
779	\\ Bank [*] (int) Data set number in a multidata file (usually 1)
780	\\ Lam (float) Specifies a wavelength in :math:`\\AA`
781	\\ Lam1 [*] (float) Specifies the primary wavelength in
782	:math:`\\AA`, used in place of Lam
783	when an :math:`\\alpha_1, \\alpha_2`
784	source is used.
785	\\ Lam2 [*] (float) Specifies the secondary wavelength in
786	:math:`\\AA`, used with Lam1
787	\\ I(L2)/I(L1) (float) Ratio of Lam2 to Lam1, used with Lam1
788	\\ Zero (float) Two-theta zero correction in degrees
789	\\ Azimuth [*] (float) Azimuthal setting angle for data recorded with differing setting angles
790	\\ U, V, W (float) Cagliotti profile coefficients
791	for Gaussian instrumental broadening, where the
792	FWHM goes as
793	:math:`U \\tan^2\\theta + V \\tan\\theta + W`
794	\\ X, Y, Z (float) Cauchy (Lorentzian) instrumental broadening coefficients
795	\\ SH/L (float) Variant of the Finger-Cox-Jephcoat asymmetric
796	peak broadening ratio. Note that this is the
797	sum of S/L and H/L where S is
798	sample height, H is the slit height and
799	L is the goniometer diameter.
800	\\ Polariz. (float) Polarization coefficient.
801	Instrument Parameters[1] (empty dict)
802	======================== =============== ===========================================================
803
804	.. _TOFPowder_table:
805
806	.. index::
807	single: Powder data TOF Instrument Parameters
808
809	-----------------------------
810	TOF Instrument Parameters
811	-----------------------------
812
813	Instrument Parameters are also placed in a list of two dicts,
814	where the keys in each dict listed below, but here for
815	time-of-flight (TOF) histograms.
816	The value for each item is a list containing three values: the initial value, the current value
817	and a refinement flag which can have a value of True, False or 0 where 0 indicates a value that
818	cannot be refined. The first and second values are floats unless otherwise noted.
819	Items not refined are noted as [*]
820
821	.. tabularcolumns:: \|l\|p{1.5in}\|p{4in}\|
822
823	======================== =============== ===========================================================
824	key sub-key explanation
825	======================== =============== ===========================================================
826	Instrument Parameters[0] Type [*] (str) Histogram type:
827	* 'PNT' for time of flight neutron
828	\\ Bank (int) Data set number in a multidata file
829	\\ 2-theta [*] (float) Nominal scattering angle for the detector
830	\\ fltPath [*] (float) Total flight path source-sample-detector
831	\\ Azimuth [*] (float) Azimuth angle for detector right hand rotation
832	from horizontal away from source
833	\\ difC,difA, (float) Diffractometer constants for conversion of d-spacing to TOF
834	difB in microseconds
835	\\ Zero (float) Zero point offset (microseconds)
836	\\ alpha (float) Exponential rise profile coefficients
837	\\ beta-0 (float) Exponential decay profile coefficients
838	beta-1
839	beta-q
840	\\ sig-0 (float) Gaussian profile coefficients
841	sig-1
842	sig-2
843	sig-q
844	\\ X,Y,Z (float) Lorentzian profile coefficients
845	Instrument Parameters[1] Pdabc (list of 4 float lists) Originally created for use in gsas as optional tables
846	of d, alp, bet, d-true; for a reflection alpha & beta are obtained via interpolation
847	from the d-spacing and these tables. The d-true column is apparently unused.
848	======================== =============== ===========================================================
849
850
851	.. _PowderRefl_table:
852
853	.. index::
854	single: Powder reflection object description
855	single: Data object descriptions; Powder Reflections
856
857	Powder Reflection Data Structure
858	--------------------------------
859
860	For every phase in a histogram, the ``Reflection Lists`` value is a dict
861	one element of which is `'RefList'`, which is a np.array containing
862	reflections. The columns in that array are documented below.
863
864	========== ====================================================
865	index explanation
866	========== ====================================================
867	0,1,2 h,k,l (float)
868	3 (int) multiplicity
869	4 (float) d-space, :math:`\\AA`
870	5 (float) pos, two-theta
871	6 (float) sig, Gaussian width
872	7 (float) gam, Lorenzian width
873	8 (float) :math:`F_{obs}^2`
874	9 (float) :math:`F_{calc}^2`
875	10 (float) reflection phase, in degrees
876	11 (float) intensity correction for reflection, this times
877	:math:`F_{obs}^2` or :math:`F_{calc}^2` gives Iobs or Icalc
878	12 (float) Preferred orientation correction
879	13 (float) Transmission (absorption correction)
880	14 (float) Extinction correction
881	========== ====================================================
882
883	.. _Xtal_table:
884
885	.. index::
886	single: Single Crystal data object description
887	single: Data object descriptions; Single crystal data
888
889	Single Crystal Tree Items
890	-------------------------
891
892	Every single crystal diffraction histogram is stored in the GSAS-II data tree
893	with a top-level entry named beginning with the string "HKLF ". The
894	diffraction data for that information are directly associated with
895	that tree item and there are a series of children to that item. The
896	routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
897	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
898	load this information into a dictionary where the child tree name is
899	used as a key, and the information in the main entry is assigned
900	a key of ``Data``, as outlined below.
901
902	.. tabularcolumns:: \|l\|l\|p{4in}\|
903
904	====================== =============== ====================================================
905	key sub-key explanation
906	====================== =============== ====================================================
907	Data \\ (dict) that contains the
908	reflection table,
909	as described in the
910	:ref:`Single Crystal Reflections
911	<XtalRefl_table>`
912	description.
913
914	Instrument Parameters \\ (list) containing two dicts where the possible
915	keys in each dict are listed below. The value
916	for most items is a list containing two values:
917	the initial value, the current value.
918	The first and second
919	values are floats unless otherwise noted.
920	\\ Lam (two floats) Specifies a wavelength in :math:`\\AA`
921	\\ Type (two str values) Histogram type :
922	* 'SXC' for constant wavelength x-ray
923	* 'SNC' for constant wavelength neutron
924	* 'SNT' for time of flight neutron
925	* 'SEC' for constant wavelength electrons (e.g. micro-ED)
926	\\ InstrName (str) A name for the instrument, used in preparing a CIF
927	wtFactor \\ (float) A weighting factor to increase or decrease
928	the leverage of data in the histogram.
929	A value of 1.0 weights the data with their
930	standard uncertainties and a larger value
931	increases the weighting of the data (equivalent
932	to decreasing the uncertainties).
933
934	hId \\ (int) The number assigned to the histogram when
935	the project is loaded or edited (can change)
936	ranId \\ (int) A random number id for the histogram
937	that does not change
938	====================== =============== ====================================================
939
940	.. _XtalRefl_table:
941
942	.. index::
943	single: Single Crystal reflection object description
944	single: Data object descriptions; Single Crystal Reflections
945
946	Single Crystal Reflection Data Structure
947	----------------------------------------
948
949	For every single crystal a histogram, the ``'Data'`` item contains
950	the structure factors as an np.array in item `'RefList'`.
951	The columns in that array are documented below.
952
953	.. tabularcolumns:: \|l\|p{4in}\|
954
955	========== ====================================================
956	index explanation
957	========== ====================================================
958	0,1,2 (float) h,k,l
959	3 (int) multiplicity
960	4 (float) d-space, :math:`\\AA`
961	5 (float) :math:`F_{obs}^2`
962	6 (float) :math:`\\sigma(F_{obs}^2)`
963	7 (float) :math:`F_{calc}^2`
964	8 (float) :math:`F_{obs}^2T`
965	9 (float) :math:`F_{calc}^2T`
966	10 (float) reflection phase, in degrees
967	11 (float) intensity correction for reflection, this times
968	:math:`F_{obs}^2` or :math:`F_{calc}^2`
969	gives Iobs or Icalc
970	========== ====================================================
971
972	.. _Image_table:
973
974	.. index::
975	image: Image data object description
976	image: Image object descriptions
977
978	Image Data Structure
979	--------------------
980
981	Every 2-dimensional image is stored in the GSAS-II data tree
982	with a top-level entry named beginning with the string "IMG ". The
983	image data are directly associated with that tree item and there
984	are a series of children to that item. The routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
985	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
986	load this information into a dictionary where the child tree name is
987	used as a key, and the information in the main entry is assigned
988	a key of ``Data``, as outlined below.
989
990	.. tabularcolumns:: \|l\|l\|p{4in}\|
991
992	====================== ====================== ====================================================
993	key sub-key explanation
994	====================== ====================== ====================================================
995	Comments \\ (list of str) Text strings extracted from the original image data
996	header or a metafile. These cannot be changed by
997	the user; it may be empty.
998	Image Controls azmthOff (float) The offset to be applied to an azimuthal
999	value. Accomodates
1000	detector orientations other than with the detector
1001	X-axis
1002	horizontal.
1003	\\ background image (list:str,float) The name of a tree item ("IMG ...") that is to be subtracted
1004	during image integration multiplied by value. It must have the same size/shape as
1005	the integrated image. NB: value < 0 for subtraction.
1006	\\ calibrant (str) The material used for determining the position/orientation
1007	of the image. The data is obtained from :func:`ImageCalibrants`
1008	and UserCalibrants.py (supplied by user).
1009	\\ calibdmin (float) The minimum d-spacing used during the last calibration run.
1010	\\ calibskip (int) The number of expected diffraction lines skipped during the last
1011	calibration run.
1012	\\ center (list:floats) The [X,Y] point in detector coordinates (mm) where the direct beam
1013	strikes the detector plane as determined by calibration. This point
1014	does not have to be within the limits of the detector boundaries.
1015	\\ centerAzm (bool) If True then the azimuth reported for the integrated slice
1016	of the image is at the center line otherwise it is at the leading edge.
1017	\\ color (str) The name of the colormap used to display the image. Default = 'Paired'.
1018	\\ cutoff (float) The minimum value of I/Ib for a point selected in a diffraction ring for
1019	calibration calculations. See pixLimit for details as how point is found.
1020	\\ DetDepth (float) Coefficient for penetration correction to distance; accounts for diffraction
1021	ring offset at higher angles. Optionally determined by calibration.
1022	\\ DetDepthRef (bool) If True then refine DetDepth during calibration/recalibration calculation.
1023	\\ distance (float) The distance (mm) from sample to detector plane.
1024	\\ ellipses (list:lists) Each object in ellipses is a list [center,phi,radii,color] where
1025	center (list) is location (mm) of the ellipse center on the detector plane, phi is the
1026	rotation of the ellipse minor axis from the x-axis, and radii are the minor & major
1027	radii of the ellipse. If radii[0] is negative then parameters describe a hyperbola. Color
1028	is the selected drawing color (one of 'b', 'g' ,'r') for the ellipse/hyperbola.
1029	\\ edgemin (float) Not used; parameter in EdgeFinder code.
1030	\\ fullIntegrate (bool) If True then integrate over full 360 deg azimuthal range.
1031	\\ GonioAngles (list:floats) The 'Omega','Chi','Phi' goniometer angles used for this image.
1032	Required for texture calculations.
1033	\\ invert_x (bool) If True display the image with the x-axis inverted.
1034	\\ invert_y (bool) If True display the image with the y-axis inverted.
1035	\\ IOtth (list:floats) The minimum and maximum 2-theta values to be used for integration.
1036	\\ LRazimuth (list:floats) The minimum and maximum azimuth values to be used for integration.
1037	\\ Oblique (list:float,bool) If True apply a detector absorption correction using the value to the
1038	intensities obtained during integration.
1039	\\ outAzimuths (int) The number of azimuth pie slices.
1040	\\ outChannels (int) The number of 2-theta steps.
1041	\\ pixelSize (list:ints) The X,Y dimensions (microns) of each pixel.
1042	\\ pixLimit (int) A box in the image with 2*pixLimit+1 edges is searched to find the maximum.
1043	This value (I) along with the minimum (Ib) in the box is reported by :func:`GSASIIimage.ImageLocalMax`
1044	and subject to cutoff in :func:`GSASIIimage.makeRing`.
1045	Locations are used to construct rings of points for calibration calcualtions.
1046	\\ PolaVal (list:float,bool) If type='SASD' and if True, apply polarization correction to intensities from
1047	integration using value.
1048	\\ rings (list:lists) Each entry is [X,Y,dsp] where X & Y are lists of x,y coordinates around a
1049	diffraction ring with the same d-spacing (dsp)
1050	\\ ring (list) The x,y coordinates of the >5 points on an inner ring
1051	selected by the user,
1052	\\ Range (list) The minimum & maximum values of the image
1053	\\ rotation (float) The angle between the x-axis and the vector about which the
1054	detector is tilted. Constrained to -180 to 180 deg.
1055	\\ SampleShape (str) Currently only 'Cylinder'. Sample shape for Debye-Scherrer experiments; used for absorption
1056	calculations.
1057	\\ SampleAbs (list: float,bool) Value of absorption coefficient for Debye-Scherrer experimnents, flag if True
1058	to cause correction to be applied.
1059	\\ setDefault (bool) If True the use the image controls values for all new images to be read. (might be removed)
1060	\\ setRings (bool) If True then display all the selected x,y ring positions (vida supra rings) used in the calibration.
1061	\\ showLines (bool) If True then isplay the integration limits to be used.
1062	\\ size (list:int) The number of pixels on the image x & y axes
1063	\\ type (str) One of 'PWDR', 'SASD' or 'REFL' for powder, small angle or reflectometry data, respectively.
1064	\\ tilt (float) The angle the detector normal makes with the incident beam; range -90 to 90.
1065	\\ wavelength (float) The radiation wavelength (:math:`\\AA`) as entered by the user
1066	(or someday obtained from the image header).
1067	Masks Arcs (list: lists) Each entry [2-theta,[azimuth[0],azimuth[1]],thickness] describes an arc mask
1068	to be excluded from integration
1069	\\ Frames (list:lists) Each entry describes the x,y points (3 or more - mm) that describe a frame outside
1070	of which is excluded from recalibration and integration. Only one frame is allowed.
1071	\\ Points (list:lists) Each entry [x,y,radius] (mm) describes an excluded spot on the image to be excluded
1072	from integration.
1073	\\ Polygons (list:lists) Each entry is a list of 3+ [x,y] points (mm) that describe a polygon on the image
1074	to be excluded from integration.
1075	\\ Rings (list: lists) Each entry [2-theta,thickness] describes a ring mask
1076	to be excluded from integration.
1077	\\ Thresholds (list:[tuple,list]) [(Imin,Imax),[Imin,Imax]] This gives lower and upper limits for points on the image to be included
1078	in integrsation. The tuple is the image intensity limits and the list are those set by the user.
1079	\\ SpotMask (dict: int & array)
1080	'esdMul'(int) number of standard deviations above mean ring intensity to mask
1081	'spotMask' (bool array) the spot mask for every pixel in image
1082
1083	Stress/Strain Sample phi (float) Sample rotation about vertical axis.
1084	\\ Sample z (float) Sample translation from the calibration sample position (for Sample phi = 0)
1085	These will be restricted by space group symmetry; result of strain fit refinement.
1086	\\ Type (str) 'True' or 'Conventional': The strain model used for the calculation.
1087	\\ d-zero (list:dict) Each item is for a diffraction ring on the image; all items are from the same phase
1088	and are used to determine the strain tensor.
1089	The dictionary items are:
1090	'Dset': (float) True d-spacing for the diffraction ring; entered by the user.
1091	'Dcalc': (float) Average calculated d-spacing determined from strain coeff.
1092	'Emat': (list: float) The strain tensor elements e11, e12 & e22 (e21=e12, rest are 0)
1093	'Esig': (list: float) Esds for Emat from fitting.
1094	'pixLimit': (int) Search range to find highest point on ring for each data point
1095	'cutoff': (float) I/Ib cutoff for searching.
1096	'ImxyObs': (list: lists) [[X],[Y]] observed points to be used for strain calculations.
1097	'ImtaObs': (list: lists) [[d],[azm]] transformed via detector calibration from ImxyObs.
1098	'ImtaCalc': (list: lists [[d],[azm]] calculated d-spacing & azimuth from fit.
1099
1100	====================== ====================== ====================================================
1101
1102	.. _parmDict_table:
1103
1104	.. index::
1105	single: Parameter dictionary
1106
1107	Parameter Dictionary
1108	-------------------------
1109
1110	The parameter dictionary contains all of the variable parameters for the refinement.
1111	The dictionary keys are the name of the parameter (<phase>:<hist>:<name>:<atom>).
1112	It is prepared in two ways. When loaded from the tree
1113	(in :meth:`GSASIIdataGUI.GSASII.MakeLSParmDict` and
1114	:meth:`GSASIIIO.ExportBaseclass.loadParmDict`),
1115	the values are lists with two elements: ``[value, refine flag]``
1116
1117	When loaded from the GPX file (in
1118	:func:`GSASIIstrMain.Refine` and :func:`GSASIIstrMain.SeqRefine`), the value in the
1119	dict is the actual parameter value (usually a float, but sometimes a
1120	letter or string flag value (such as I or A for iso/anisotropic).
1121
1122	Texture implementation
1123	------------------------------
1124
1125	There are two different places where texture can be treated in GSAS-II.
1126	One is for mitigating the effects of texture in a structural refinement.
1127	The other is for texture characterization.
1128
1129	For reducing the effect of texture in a structural refinement
1130	there are entries labeled preferred orientation in each phase's
1131	data tab. Two different approaches can be used for this, the March-Dollase
1132	model and spherical harmonics.
1133
1134	For the March-Dollase model, one axis in reciprocal space is designated as
1135	unique (defaulting to the 001 axis) and reflections are corrected
1136	according to the angle they make with this axis depending on
1137	the March-Dollase ratio. (If unity, no correction is made).
1138	The ratio can be greater than one or less than one depending on if
1139	crystallites oriented along the designated axis are
1140	overrepresented or underrepresented. For most crystal systems there is an
1141	obvious choice for the direction of the unique axis and then only a single
1142	term needs to be refined. If the number is close to 1, then the correction
1143	is not needed.
1144
1145	The second method for reducing the effect of texture in a structural
1146	refinement is to create a crystallite orientation probability surface as an
1147	expansion in terms spherical harmonic functions. Only functions consistent with
1148	cylindrical diffraction suymmetry and having texture symmetry
1149	consistent with the Laue class of phase are used and are allowed,
1150	so the higher the symmetry the fewer terms that are available for a given spherical harmonics order.
1151	To use this correction, select the lowest order that provides
1152	refinable terms and perform a refinement. If the texture index remains close to
1153	one, then the correction is not needed. If a significant improvement is
1154	noted in the profile Rwp, one may wish to see if a higher order expansion
1155	gives an even larger improvement.
1156
1157	To characterize texture in a material, generally one needs data collected with the
1158	sample at multiple orientations or, for TOF, with detectors at multiple
1159	locations around the sample. In this case the detector orientation is given in
1160	each histogram's Sample Parameters and the sample's orientation is described
1161	with the Euler angles specifed on the phase's Texture tab, which is also
1162	where the texture type (cylindrical, rolling,...) and the spherical
1163	harmonic order is selected. This should not be used with a single dataset and
1164	should not be used if the preferred orientations corrections are used.
1165
1166	The coordinate system used for texture characterization is defined where
1167	the sample coordinates (Psi, gamma) are defined with an instrument coordinate
1168	system (I, J, K) such that K is normal to the diffraction plane and J is coincident with the
1169	direction of the incident radiation beam toward the source. We further define
1170	a standard set of right-handed goniometer eulerian angles (Omega, Chi, Phi) so that Omega and Phi are
1171	rotations about K and Chi is a rotation about J when Omega = 0. Finally, as the sample
1172	may be mounted so that the sample coordinate system (Is, Js, Ks) does not coincide with
1173	the instrument coordinate system (I, J, K), we define three eulerian sample rotation angles
1174	(Omega-s, Chi-s, Phi-s) that describe the rotation from (Is, Js, Ks) to (I, J, K). The sample rotation
1175	angles are defined so that with the goniometer angles at zero Omega-s and Phi-s are rotations
1176	about K and Chi-s is a rotation about J.
1177
1178	Three typical examples:
1179
1180	1) Bragg-Brentano laboratory diffractometer: Chi=0
1181	2) Debye-Scherrer counter detector; sample capillary axis perpendicular to diffraction plane: Chi=90
1182	3) Debye-Scherrer 2D area detector positioned directly behind sample; sample capillary axis horizontal; Chi=0
1183
1184	NB: The area detector azimuthal angle will equal 0 in horizontal plane to right as viewed from x-ray source and will equal
1185	90 at vertical "up" direction.
1186
1187	ISODISTORT implementation
1188	------------------------------
1189
1190	CIFs prepared with the ISODISTORT web site
1191	https://stokes.byu.edu/iso/isodistort_version5.6.1/isodistort.php
1192	[B. J. Campbell, H. T. Stokes, D. E. Tanner, and D. M. Hatch, "ISODISPLACE: An Internet Tool for Exploring Structural Distortions."
1193	J. Appl. Cryst. 39, 607-614 (2006).] can be read into GSAS-II using import CIF. This will cause constraints to be established for
1194	structural distortion modes read from the CIF. At present, of the five types of modes only displacive(``_iso_displacivemode``...)
1195	and occupancy (``_iso_occupancymode``...) are processed. Not yet processed: ``_iso_magneticmode``...,
1196	``_iso_rotationalmode``... & ``_iso_strainmode``...
1197
1198	The CIF importer :mod:`G2phase_CIF` implements class :class:`G2phase_CIF.CIFPhaseReader` which offers two methods associated
1199	with ISODISTORT (ID) input. Method :meth:`G2phase_CIF.CIFPhaseReader.ISODISTORT_test` checks to see if a CIF block contains
1200	the loops with ``_iso_displacivemode_label`` or ``_iso_occupancymode_label`` items. If so, method
1201	:meth:`G2phase_CIF.CIFPhaseReader.ISODISTORT_proc` is called to read and interpret them. The results are placed into the
1202	reader object's ``.Phase`` class variable as a dict item with key ``'ISODISTORT'``.
1203
1204	Note that each mode ID has a long label with a name such as Pm-3m[1/2,1/2,1/2]R5+(a,a,0)[La:b:dsp]T1u(a). Function
1205	:func:`G2phase_CIF.ISODISTORT_shortLbl` is used to create a short name for this, such as R5_T1u(a) which is made unique
1206	by addition of _n if the short name is duplicated. As each mode is processed, a constraint corresponding to that mode is
1207	created and is added to list in the reader object's ``.Constraints`` class variable. Items placed into that list can either
1208	be a list, which corresponds to a function (new var) type :ref:`constraint definition <Constraints_table>` entry, or an item
1209	can be a dict, which provides help information for each constraint.
1210
1211	------------------------------
1212	Displacive modes
1213	------------------------------
1214
1215	The coordinate variables, as named by ISODISTORT, are placed in ``.Phase['ISODISTORT']['IsoVarList']`` and the
1216	corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed in ``.Phase['ISODISTORT']['G2VarList']``.
1217	The mode variables, as named by ISODISTORT, are placed in ``.Phase['ISODISTORT']['IsoModeList']`` and the
1218	corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed in ``.Phase['ISODISTORT']['G2ModeList']``.
1219	[Use ``str(G2VarObj)`` to get the variable name from the G2VarObj object, but note that the phase number, n, for the prefix
1220	"n::" cannot be determined as the phase number is not yet assigned.]
1221
1222	Displacive modes are a bit complex in that they relate to delta displacements, relative to an offset value for each coordinate,
1223	and because the modes are normalized. While GSAS-II also uses displacements, these are added to the coordinates after
1224	each refinement cycle and then the delta values are set to zero.
1225	ISODISTORT uses fixed offsets (subtracted from the actual position
1226	to obtain the delta values) that are taken from the parent structure coordinate and the initial offset value
1227	(in ``_iso_deltacoordinate_value``) and these are placed in
1228	``.Phase['ISODISTORT']['G2coordOffset']`` in the same order as ``.Phase['ISODISTORT']['G2ModeList']``,
1229	``.Phase['ISODISTORT']['IsoVarList']`` and ''.Phase[ISODISTORT']['G2parentCoords']''.'
1230
1231	The normalization factors (which the delta values are divided by)
1232	are taken from ``_iso_displacivemodenorm_value`` and are placed in ``.Phase['ISODISTORT']['NormList']`` in the same
1233	order as as ``...['IsoModeList']`` and ``...['G2ModeList']``.
1234
1235	The CIF contains a sparse matrix, from the ``loop_`` containing ``_iso_displacivemodematrix_value`` which provides the equations
1236	for determining the mode values from the coordinates, that matrix is placed in ``.Phase['ISODISTORT']['Mode2VarMatrix']``.
1237	The matrix is inverted to produce ``.Phase['ISODISTORT']['Var2ModeMatrix']``, which determines how to compute the
1238	mode values from the delta coordinate values. These values are used for the in :func:`GSASIIconstrGUI.ShowIsoDistortCalc`,
1239	which shows coordinate and mode values, the latter with s.u. values.
1240
1241	------------------------------
1242	Occupancy modes
1243	------------------------------
1244
1245
1246	The delta occupancy variables, as named by ISODISTORT, are placed in
1247	``.Phase['ISODISTORT']['OccVarList']`` and the corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed
1248	in ``.Phase['ISODISTORT']['G2OccVarList']``. The mode variables, as named by ISODISTORT, are placed in
1249	``.Phase['ISODISTORT']['OccModeList']`` and the corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed
1250	in ``.Phase['ISODISTORT']['G2OccModeList']``.
1251
1252	Occupancy modes, like Displacive modes, are also refined as delta values. However, GSAS-II directly refines the fractional
1253	occupancies. Offset values for each atom, are taken from ``_iso_occupancy_formula`` and are placed in
1254	``.Phase['ISODISTORT']['ParentOcc]``. (Offset values are subtracted from the actual position to obtain the delta values.)
1255	Modes are normalized (where the mode values are divided by the normalization factor) are taken from ``_iso_occupancymodenorm_value``
1256	and are placed in ``.Phase['ISODISTORT']['OccNormList']`` in the same order as as ``...['OccModeList']`` and
1257	``...['G2OccModeList']``.
1258
1259	The CIF contains a sparse matrix, from the ``loop_`` containing ``_iso_occupancymodematrix_value``, which provides the
1260	equations for determining the mode values from the coordinates. That matrix is placed in ``.Phase['ISODISTORT']['Occ2VarMatrix']``.
1261	The matrix is inverted to produce ``.Phase['ISODISTORT']['Var2OccMatrix']``, which determines how to compute the
1262	mode values from the delta coordinate values.
1263
1264
1265	------------------------------
1266	Mode Computations
1267	------------------------------
1268
1269	Constraints are processed after the CIF has been read in :meth:`GSASIIdataGUI.GSASII.OnImportPhase` or
1270	:meth:`GSASIIscriptable.G2Project.add_phase` by moving them from the reader object's ``.Constraints``
1271	class variable to the Constraints tree entry's ['Phase'] list (for list items defining constraints) or
1272	the Constraints tree entry's ['_Explain'] dict (for dict items defining constraint help information)
1273
1274	The information in ``.Phase['ISODISTORT']`` is used in :func:`GSASIIconstrGUI.ShowIsoDistortCalc` which shows coordinate and mode
1275	values, the latter with s.u. values. This can be called from the Constraints and Phase/Atoms tree items.
1276
1277	Before each refinement, constraints are processed as :ref:`described elsewhere <Constraints_processing>`. After a refinement
1278	is complete, :func:`GSASIIstrIO.PrintIndependentVars` shows the shifts and s.u.'s on the refined modes,
1279	using GSAS-II values, but :func:`GSASIIstrIO.PrintISOmodes` prints the ISODISTORT modes as computed in the web site.
1280
1281
1282	.. _ParameterLimits:
1283
1284	.. index::
1285	single: Parameter limits
1286
1287	Parameter Limits
1288	------------------------------
1289
1290	One of the most often requested "enhancements" for GSAS-II would be the inclusion
1291	of constraints to force parameters such as occupancies or Uiso values to stay within
1292	expected ranges. While it is possible for users to supply their own restraints that would
1293	perform this by supplying an appropriate expression with the "General" restraints, the
1294	GSAS-II authors do not feel that use of restraints or constraints are a good solution for
1295	this common problem where parameters refine to non-physical values. This is because when
1296	this occurs, most likely one of the following cases is occurring:
1297
1298	#. there is a significant problem
1299	with the model, for example for an x-ray fit if an O atom is placed where a S is actually
1300	present, the Uiso will refine artificially small or the occupancy much larger than unity
1301	to try to compensate for the missing electrons; or
1302
1303	#. the data are simply insensitive
1304	to the parameter or combination of parameters, for example unless very high-Q data
1305	are included, the effects of a occupancy and Uiso value can have compensating effects,
1306	so an assumption must be made; likewise, with neutron data natural-abundance V atoms
1307	are nearly invisible due to weak coherent scattering. No parameters can be fit for a
1308	V atom with neutrons.
1309
1310	#. the parameter is non-physical (such as a negative Uiso value) but within
1311	two sigma (sigma = standard uncertainty, aka e.s.d.) of a reasonable value,
1312	in which case the
1313	value is not problematic as it is experimentally indistinguishable from an
1314	expected value.
1315
1316	#. there is a systematic problem with the data (experimental error)
1317
1318	In all these cases, this situation needs to be reviewed by a crystallographer to decide
1319	how to best determine a structural model for these data. An implementation with a constraint
1320	or restraint is likely to simply hide the problem from the user, making it more probable
1321	that a poor model choice is obtained.
1322
1323	What GSAS-II does implement is to allow users to specify ranges for parameters
1324	that works by disabling
1325	refinement of parameters that refine beyond either a lower limit or an upper limit, where
1326	either or both may be optionally specified. Parameters limits are specified in the Controls
1327	tree entry in dicts named as ``Controls['parmMaxDict']`` and ``Controls['parmMinDict']``, where
1328	the keys are :class:`G2VarObj` objects corresponding to standard GSAS-II variable
1329	(see :func:`getVarDescr` and :func:`CompileVarDesc`) names, where a
1330	wildcard ('*') may optionally be used for histogram number or atom number
1331	(phase number is intentionally not allowed as a wildcard as it makes little sense
1332	to group the same parameter together different phases). Note
1333	that :func:`prmLookup` is used to see if a name matches a wildcard. The upper or lower limit
1334	is placed into these dicts as a float value. These values can be edited using the window
1335	created by the Calculate/"View LS parms" menu command or in scripting with the
1336	:meth:`GSASIIscriptable.G2Project.set_Controls` function.
1337	In the GUI, a checkbox labeled "match all histograms/atoms" is used to insert a wildcard
1338	into the appropriate part of the variable name.
1339
1340	When a refinement is conducted, routine :func:`GSASIIstrMain.dropOOBvars` is used to
1341	find parameters that have refined to values outside their limits. If this occurs, the parameter
1342	is set to the limiting value and the variable name is added to a list of frozen variables
1343	(as a :class:`G2VarObj` objects) kept in a list in the
1344	``Controls['parmFrozen']`` dict. In a sequential refinement, this is kept separate for
1345	each histogram as a list in
1346	``Controls['parmFrozen'][histogram]`` (where the key is the histogram name) or as a list in
1347	``Controls['parmFrozen']['FrozenList']`` for a non-sequential fit.
1348	This allows different variables
1349	to be frozen in each section of a sequential fit.
1350	Frozen parameters are not included in refinements through removal from the
1351	list of parameters to be refined (``varyList``) in :func:`GSASIIstrMain.Refine` or
1352	:func:`GSASIIstrMain.SeqRefine`.
1353	The data window for the Controls tree item shows the number of Frozen variables and
1354	the individual variables can be viewed with the Calculate/"View LS parms" menu window or
1355	obtained with :meth:`GSASIIscriptable.G2Project.get_Frozen`.
1356	Once a variable is frozen, it will not be refined in any
1357	future refinements unless the the variable is removed (manually) from the list. This can also
1358	be done with the Calculate/"View LS parms" menu window or
1359	:meth:`GSASIIscriptable.G2Project.set_Frozen`.
1360
1361
1362	.. seealso::
1363	:class:`G2VarObj`
1364	:func:`getVarDescr`
1365	:func:`CompileVarDesc`
1366	:func:`prmLookup`
1367	:class:`GSASIIctrlGUI.ShowLSParms`
1368	:class:`GSASIIctrlGUI.VirtualVarBox`
1369	:func:`GSASIIstrIO.SetUsedHistogramsAndPhases`
1370	:func:`GSASIIstrIO.SaveUpdatedHistogramsAndPhases`
1371	:func:`GSASIIstrIO.SetSeqResult`
1372	:func:`GSASIIstrMain.dropOOBvars`
1373	:meth:`GSASIIscriptable.G2Project.set_Controls`
1374	:meth:`GSASIIscriptable.G2Project.get_Frozen`
1375	:meth:`GSASIIscriptable.G2Project.set_Frozen`
1376
1377	Classes and routines
1378	----------------------
1379
1380	'''
1381	from __future__ import division, print_function
1382	import platform
1383	import re
1384	import random as ran
1385	import sys
1386	import os.path
1387	if '2' in platform.python_version_tuple()[0]:
1388	import cPickle
1389	else:
1390	import pickle as cPickle
1391	import GSASIIpath
1392	import GSASIImath as G2mth
1393	import GSASIIspc as G2spc
1394	import numpy as np
1395
1396	GSASIIpath.SetVersionNumber("$Revision: 5307 $")
1397
1398	DefaultControls = {
1399	'deriv type':'analytic Hessian',
1400	'min dM/M':0.001,'shift factor':1.,'max cyc':3,'F**2':False,'SVDtol':1.e-6,
1401	'UsrReject':{'minF/sig':0.,'MinExt':0.01,'MaxDF/F':100.,'MaxD':500.,'MinD':0.05},
1402	'Copy2Next':False,'Reverse Seq':False,'HatomFix':False,
1403	'Author':'no name','newLeBail':False,
1404	'FreePrm1':'Sample humidity (%)',
1405	'FreePrm2':'Sample voltage (V)',
1406	'FreePrm3':'Applied load (MN)',
1407	'ShowCell':False,
1408	}
1409	'''Values to be used as defaults for the initial contents of the ``Controls``
1410	data tree item.
1411	'''
1412	def StripUnicode(string,subs='.'):
1413	'''Strip non-ASCII characters from strings
1414
1415	:param str string: string to strip Unicode characters from
1416	:param str subs: character(s) to place into string in place of each
1417	Unicode character. Defaults to '.'
1418
1419	:returns: a new string with only ASCII characters
1420	'''
1421	s = ''
1422	for c in string:
1423	if ord(c) < 128:
1424	s += c
1425	else:
1426	s += subs
1427	return s
1428	# return s.encode('ascii','replace')
1429
1430	def MakeUniqueLabel(lbl,labellist):
1431	'''Make sure that every a label is unique against a list by adding
1432	digits at the end until it is not found in list.
1433
1434	:param str lbl: the input label
1435	:param list labellist: the labels that have already been encountered
1436	:returns: lbl if not found in labellist or lbl with ``_1-9`` (or
1437	``_10-99``, etc.) appended at the end
1438	'''
1439	lbl = StripUnicode(lbl.strip(),'_')
1440	if not lbl: # deal with a blank label
1441	lbl = '_1'
1442	if lbl not in labellist:
1443	labellist.append(lbl)
1444	return lbl
1445	i = 1
1446	prefix = lbl
1447	if '_' in lbl:
1448	prefix = lbl[:lbl.rfind('_')]
1449	suffix = lbl[lbl.rfind('_')+1:]
1450	try:
1451	i = int(suffix)+1
1452	except: # suffix could not be parsed
1453	i = 1
1454	prefix = lbl
1455	while prefix+'_'+str(i) in labellist:
1456	i += 1
1457	else:
1458	lbl = prefix+'_'+str(i)
1459	labellist.append(lbl)
1460	return lbl
1461
1462	PhaseIdLookup = {}
1463	'''dict listing phase name and random Id keyed by sequential phase index as a str;
1464	best to access this using :func:`LookupPhaseName`
1465	'''
1466	PhaseRanIdLookup = {}
1467	'''dict listing phase sequential index keyed by phase random Id;
1468	best to access this using :func:`LookupPhaseId`
1469	'''
1470	HistIdLookup = {}
1471	'''dict listing histogram name and random Id, keyed by sequential histogram index as a str;
1472	best to access this using :func:`LookupHistName`
1473	'''
1474	HistRanIdLookup = {}
1475	'''dict listing histogram sequential index keyed by histogram random Id;
1476	best to access this using :func:`LookupHistId`
1477	'''
1478	AtomIdLookup = {}
1479	'''dict listing for each phase index as a str, the atom label and atom random Id,
1480	keyed by atom sequential index as a str;
1481	best to access this using :func:`LookupAtomLabel`
1482	'''
1483	AtomRanIdLookup = {}
1484	'''dict listing for each phase the atom sequential index keyed by atom random Id;
1485	best to access this using :func:`LookupAtomId`
1486	'''
1487	ShortPhaseNames = {}
1488	'''a dict containing a possibly shortened and when non-unique numbered
1489	version of the phase name. Keyed by the phase sequential index.
1490	'''
1491	ShortHistNames = {}
1492	'''a dict containing a possibly shortened and when non-unique numbered
1493	version of the histogram name. Keyed by the histogram sequential index.
1494	'''
1495
1496	#VarDesc = {} # removed 1/30/19 BHT as no longer needed (I think)
1497	#''' This dictionary lists descriptions for GSAS-II variables,
1498	#as set in :func:`CompileVarDesc`. See that function for a description
1499	#for how keys and values are written.
1500	#'''
1501
1502	reVarDesc = {}
1503	''' This dictionary lists descriptions for GSAS-II variables where
1504	keys are compiled regular expressions that will match the name portion
1505	of a parameter name. Initialized in :func:`CompileVarDesc`.
1506	'''
1507
1508	reVarStep = {}
1509	''' This dictionary lists the preferred step size for numerical
1510	derivative computation w/r to a GSAS-II variable. Keys are compiled
1511	regular expressions and values are the step size for that parameter.
1512	Initialized in :func:`CompileVarDesc`.
1513	'''
1514	# create a default space group object for P1; N.B. fails when building documentation
1515	try:
1516	P1SGData = G2spc.SpcGroup('P 1')[1] # data structure for default space group
1517	except:
1518	pass
1519
1520	def GetPhaseNames(fl):
1521	''' Returns a list of phase names found under 'Phases' in GSASII gpx file
1522	NB: there is another one of these in GSASIIstrIO.py that uses the gpx filename
1523
1524	:param file fl: opened .gpx file
1525	:return: list of phase names
1526	'''
1527	PhaseNames = []
1528	while True:
1529	try:
1530	data = cPickle.load(fl)
1531	except EOFError:
1532	break
1533	datum = data[0]
1534	if 'Phases' == datum[0]:
1535	for datus in data[1:]:
1536	PhaseNames.append(datus[0])
1537	fl.seek(0) #reposition file
1538	return PhaseNames
1539
1540	def SetNewPhase(Name='New Phase',SGData=None,cell=None,Super=None):
1541	'''Create a new phase dict with default values for various parameters
1542
1543	:param str Name: Name for new Phase
1544
1545	:param dict SGData: space group data from :func:`GSASIIspc:SpcGroup`;
1546	defaults to data for P 1
1547
1548	:param list cell: unit cell parameter list; defaults to
1549	[1.0,1.0,1.0,90.,90,90.,1.]
1550
1551	'''
1552	if SGData is None: SGData = P1SGData
1553	if cell is None: cell=[1.0,1.0,1.0,90.,90.,90.,1.]
1554	phaseData = {
1555	'ranId':ran.randint(0,sys.maxsize),
1556	'General':{
1557	'Name':Name,
1558	'Type':'nuclear',
1559	'Modulated':False,
1560	'AtomPtrs':[3,1,7,9],
1561	'SGData':SGData,
1562	'Cell':[False,]+cell,
1563	'Pawley dmin':1.0,
1564	'Data plot type':'None',
1565	'SH Texture':{
1566	'Order':0,
1567	'Model':'cylindrical',
1568	'Sample omega':[False,0.0],
1569	'Sample chi':[False,0.0],
1570	'Sample phi':[False,0.0],
1571	'SH Coeff':[False,{}],
1572	'SHShow':False,
1573	'PFhkl':[0,0,1],
1574	'PFxyz':[0,0,1],
1575	'PlotType':'Pole figure',
1576	'Penalty':[['',],0.1,False,1.0]}},
1577	'Atoms':[],
1578	'Drawing':{},
1579	'Histograms':{},
1580	'Pawley ref':[],
1581	'RBModels':{},
1582	}
1583	if Super and Super.get('Use',False):
1584	phaseData['General'].update({'Modulated':True,'Super':True,'SuperSg':Super['ssSymb']})
1585	phaseData['General']['SSGData'] = G2spc.SSpcGroup(SGData,Super['ssSymb'])[1]
1586	phaseData['General']['SuperVec'] = [Super['ModVec'],False,Super['maxH']]
1587
1588	return phaseData
1589
1590	def ReadCIF(URLorFile):
1591	'''Open a CIF, which may be specified as a file name or as a URL using PyCifRW
1592	(from James Hester).
1593	The open routine gets confused with DOS names that begin with a letter and colon
1594	"C:\\dir\" so this routine will try to open the passed name as a file and if that
1595	fails, try it as a URL
1596
1597	:param str URLorFile: string containing a URL or a file name. Code will try first
1598	to open it as a file and then as a URL.
1599
1600	:returns: a PyCifRW CIF object.
1601	'''
1602	import CifFile as cif # PyCifRW from James Hester
1603
1604	# alternate approach:
1605	#import urllib
1606	#ciffile = 'file:'+urllib.pathname2url(filename)
1607
1608	try:
1609	fp = open(URLorFile,'r')
1610	cf = cif.ReadCif(fp)
1611	fp.close()
1612	return cf
1613	except IOError:
1614	return cif.ReadCif(URLorFile)
1615
1616	def TestIndexAll():
1617	'''Test if :func:`IndexAllIds` has been called to index all phases and
1618	histograms (this is needed before :func:`G2VarObj` can be used.
1619
1620	:returns: Returns True if indexing is needed.
1621	'''
1622	if PhaseIdLookup or AtomIdLookup or HistIdLookup:
1623	return False
1624	return True
1625
1626	def IndexAllIds(Histograms,Phases):
1627	'''Scan through the used phases & histograms and create an index
1628	to the random numbers of phases, histograms and atoms. While doing this,
1629	confirm that assigned random numbers are unique -- just in case lightning
1630	strikes twice in the same place.
1631
1632	Note: this code assumes that the atom random Id (ranId) is the last
1633	element each atom record.
1634
1635	This is called when phases & histograms are looked up
1636	in these places (only):
1637
1638	* :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` (which loads the histograms and phases from a GPX file),
1639	* :meth:`~GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree` (which does the same thing but from the data tree.)
1640	* :meth:`~GSASIIdataGUI.GSASII.OnFileClose` (clears out an old project)
1641
1642	Note that globals :data:`PhaseIdLookup` and :data:`PhaseRanIdLookup` are
1643	also set in :func:`AddPhase2Index` to temporarily assign a phase number
1644	as a phase is being imported.
1645
1646	TODO: do we need a lookup for rigid body variables?
1647	'''
1648	# process phases and atoms
1649	PhaseIdLookup.clear()
1650	PhaseRanIdLookup.clear()
1651	AtomIdLookup.clear()
1652	AtomRanIdLookup.clear()
1653	ShortPhaseNames.clear()
1654	for ph in Phases:
1655	cx,ct,cs,cia = Phases[ph]['General']['AtomPtrs']
1656	ranId = Phases[ph]['ranId']
1657	while ranId in PhaseRanIdLookup:
1658	# Found duplicate random Id! note and reassign
1659	print ("\n\n*** Phase "+str(ph)+" has repeated ranId. Fixing.\n")
1660	Phases[ph]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1661	pId = str(Phases[ph]['pId'])
1662	PhaseIdLookup[pId] = (ph,ranId)
1663	PhaseRanIdLookup[ranId] = pId
1664	shortname = ph #[:10]
1665	while shortname in ShortPhaseNames.values():
1666	shortname = ph[:8] + ' ('+ pId + ')'
1667	ShortPhaseNames[pId] = shortname
1668	AtomIdLookup[pId] = {}
1669	AtomRanIdLookup[pId] = {}
1670	for iatm,at in enumerate(Phases[ph]['Atoms']):
1671	ranId = at[cia+8]
1672	while ranId in AtomRanIdLookup[pId]: # check for dups
1673	print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1674	at[cia+8] = ranId = ran.randint(0,sys.maxsize)
1675	AtomRanIdLookup[pId][ranId] = str(iatm)
1676	if Phases[ph]['General']['Type'] == 'macromolecular':
1677	label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1678	else:
1679	label = at[ct-1]
1680	AtomIdLookup[pId][str(iatm)] = (label,ranId)
1681	# process histograms
1682	HistIdLookup.clear()
1683	HistRanIdLookup.clear()
1684	ShortHistNames.clear()
1685	for hist in Histograms:
1686	ranId = Histograms[hist]['ranId']
1687	while ranId in HistRanIdLookup:
1688	# Found duplicate random Id! note and reassign
1689	print ("\n\n*** Histogram "+str(hist)+" has repeated ranId. Fixing.\n")
1690	Histograms[hist]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1691	hId = str(Histograms[hist]['hId'])
1692	HistIdLookup[hId] = (hist,ranId)
1693	HistRanIdLookup[ranId] = hId
1694	shortname = hist[:15]
1695	while shortname in ShortHistNames.values():
1696	shortname = hist[:11] + ' ('+ hId + ')'
1697	ShortHistNames[hId] = shortname
1698
1699	def AddPhase2Index(rdObj,filename):
1700	'''Add a phase to the index during reading
1701	Used where constraints are generated during import (ISODISTORT CIFs)
1702	'''
1703	ranId = rdObj.Phase['ranId']
1704	ph = 'from '+filename # phase is not named yet
1705	if ranId in PhaseRanIdLookup: return
1706	maxph = -1
1707	for r in PhaseRanIdLookup:
1708	maxph = max(maxph,int(PhaseRanIdLookup[r]))
1709	PhaseRanIdLookup[ranId] = pId = str(maxph + 1)
1710	PhaseIdLookup[pId] = (ph,ranId)
1711	shortname = 'from '+ os.path.splitext((os.path.split(filename))[1])[0]
1712	while shortname in ShortPhaseNames.values():
1713	shortname = ph[:8] + ' ('+ pId + ')'
1714	ShortPhaseNames[pId] = shortname
1715	AtomIdLookup[pId] = {}
1716	AtomRanIdLookup[pId] = {}
1717	for iatm,at in enumerate(rdObj.Phase['Atoms']):
1718	ranId = at[-1]
1719	while ranId in AtomRanIdLookup[pId]: # check for dups
1720	print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1721	at[-1] = ranId = ran.randint(0,sys.maxsize)
1722	AtomRanIdLookup[pId][ranId] = str(iatm)
1723	#if Phases[ph]['General']['Type'] == 'macromolecular':
1724	# label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1725	#else:
1726	# label = at[ct-1]
1727	label = at[0]
1728	AtomIdLookup[pId][str(iatm)] = (label,ranId)
1729
1730	def LookupAtomId(pId,ranId):
1731	'''Get the atom number from a phase and atom random Id
1732
1733	:param int/str pId: the sequential number of the phase
1734	:param int ranId: the random Id assigned to an atom
1735
1736	:returns: the index number of the atom (str)
1737	'''
1738	if not AtomRanIdLookup:
1739	raise Exception('Error: LookupAtomId called before IndexAllIds was run')
1740	if pId is None or pId == '':
1741	raise KeyError('Error: phase is invalid (None or blank)')
1742	pId = str(pId)
1743	if pId not in AtomRanIdLookup:
1744	raise KeyError('Error: LookupAtomId does not have phase '+pId)
1745	if ranId not in AtomRanIdLookup[pId]:
1746	raise KeyError('Error: LookupAtomId, ranId '+str(ranId)+' not in AtomRanIdLookup['+pId+']')
1747	return AtomRanIdLookup[pId][ranId]
1748
1749	def LookupAtomLabel(pId,index):
1750	'''Get the atom label from a phase and atom index number
1751
1752	:param int/str pId: the sequential number of the phase
1753	:param int index: the index of the atom in the list of atoms
1754
1755	:returns: the label for the atom (str) and the random Id of the atom (int)
1756	'''
1757	if not AtomIdLookup:
1758	raise Exception('Error: LookupAtomLabel called before IndexAllIds was run')
1759	if pId is None or pId == '':
1760	raise KeyError('Error: phase is invalid (None or blank)')
1761	pId = str(pId)
1762	if pId not in AtomIdLookup:
1763	raise KeyError('Error: LookupAtomLabel does not have phase '+pId)
1764	if index not in AtomIdLookup[pId]:
1765	raise KeyError('Error: LookupAtomLabel, ranId '+str(index)+' not in AtomRanIdLookup['+pId+']')
1766	return AtomIdLookup[pId][index]
1767
1768	def LookupPhaseId(ranId):
1769	'''Get the phase number and name from a phase random Id
1770
1771	:param int ranId: the random Id assigned to a phase
1772	:returns: the sequential Id (pId) number for the phase (str)
1773	'''
1774	if not PhaseRanIdLookup:
1775	raise Exception('Error: LookupPhaseId called before IndexAllIds was run')
1776	if ranId not in PhaseRanIdLookup:
1777	raise KeyError('Error: LookupPhaseId does not have ranId '+str(ranId))
1778	return PhaseRanIdLookup[ranId]
1779
1780	def LookupPhaseName(pId):
1781	'''Get the phase number and name from a phase Id
1782
1783	:param int/str pId: the sequential assigned to a phase
1784	:returns: (phase,ranId) where phase is the name of the phase (str)
1785	and ranId is the random # id for the phase (int)
1786	'''
1787	if not PhaseIdLookup:
1788	raise Exception('Error: LookupPhaseName called before IndexAllIds was run')
1789	if pId is None or pId == '':
1790	raise KeyError('Error: phase is invalid (None or blank)')
1791	pId = str(pId)
1792	if pId not in PhaseIdLookup:
1793	raise KeyError('Error: LookupPhaseName does not have index '+pId)
1794	return PhaseIdLookup[pId]
1795
1796	def LookupHistId(ranId):
1797	'''Get the histogram number and name from a histogram random Id
1798
1799	:param int ranId: the random Id assigned to a histogram
1800	:returns: the sequential Id (hId) number for the histogram (str)
1801	'''
1802	if not HistRanIdLookup:
1803	raise Exception('Error: LookupHistId called before IndexAllIds was run')
1804	if ranId not in HistRanIdLookup:
1805	raise KeyError('Error: LookupHistId does not have ranId '+str(ranId))
1806	return HistRanIdLookup[ranId]
1807
1808	def LookupHistName(hId):
1809	'''Get the histogram number and name from a histogram Id
1810
1811	:param int/str hId: the sequential assigned to a histogram
1812	:returns: (hist,ranId) where hist is the name of the histogram (str)
1813	and ranId is the random # id for the histogram (int)
1814	'''
1815	if not HistIdLookup:
1816	raise Exception('Error: LookupHistName called before IndexAllIds was run')
1817	if hId is None or hId == '':
1818	raise KeyError('Error: histogram is invalid (None or blank)')
1819	hId = str(hId)
1820	if hId not in HistIdLookup:
1821	raise KeyError('Error: LookupHistName does not have index '+hId)
1822	return HistIdLookup[hId]
1823
1824	def fmtVarDescr(varname):
1825	'''Return a string with a more complete description for a GSAS-II variable
1826
1827	:param str varname: A full G2 variable name with 2 or 3 or 4
1828	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1829
1830	:returns: a string with the description
1831	'''
1832	s,l = VarDescr(varname)
1833	return s+": "+l
1834
1835	def VarDescr(varname):
1836	'''Return two strings with a more complete description for a GSAS-II variable
1837
1838	:param str name: A full G2 variable name with 2 or 3 or 4
1839	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1840
1841	:returns: (loc,meaning) where loc describes what item the variable is mapped
1842	(phase, histogram, etc.) and meaning describes what the variable does.
1843	'''
1844
1845	# special handling for parameter names without a colon
1846	# for now, assume self-defining
1847	if varname.find(':') == -1:
1848	return "Global",varname
1849
1850	l = getVarDescr(varname)
1851	if not l:
1852	return ("invalid variable name ("+str(varname)+")!"),""
1853	# return "invalid variable name!",""
1854
1855	if not l[-1]:
1856	l[-1] = "(variable needs a definition! Set it in CompileVarDesc)"
1857
1858	if len(l) == 3: #SASD variable name!
1859	s = 'component:'+l[1]
1860	return s,l[-1]
1861	s = ""
1862	if l[0] is not None and l[1] is not None: # HAP: keep short
1863	if l[2] == "Scale": # fix up ambigous name
1864	l[5] = "Phase fraction"
1865	if l[0] == '*':
1866	lbl = 'Seq. ref.'
1867	else:
1868	lbl = ShortPhaseNames.get(l[0],'? #'+str(l[0]))
1869	if l[1] == '*':
1870	hlbl = 'Seq. ref.'
1871	else:
1872	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1873	if hlbl[:4] == 'HKLF':
1874	hlbl = 'Xtl='+hlbl[5:]
1875	elif hlbl[:4] == 'PWDR':
1876	hlbl = 'Pwd='+hlbl[5:]
1877	else:
1878	hlbl = 'Hist='+hlbl
1879	s = "Ph="+str(lbl)+" * "+str(hlbl)
1880	else:
1881	if l[2] == "Scale": # fix up ambigous name: must be scale factor, since not HAP
1882	l[5] = "Scale factor"
1883	if l[2] == 'Back': # background parameters are "special", alas
1884	s = 'Hist='+ShortHistNames.get(l[1],'? #'+str(l[1]))
1885	l[-1] += ' #'+str(l[3])
1886	elif l[4] is not None: # rigid body parameter or modulation parm
1887	lbl = ShortPhaseNames.get(l[0],'phase?')
1888	if 'RB' in l[2]: #rigid body parm
1889	s = "RB body #"+str(l[3])+" (type "+str(l[4])+") in "+str(lbl) + ','
1890	else: #modulation parm
1891	s = 'Atom %s wave %s in %s'%(LookupAtomLabel(l[0],l[3])[0],l[4],lbl)
1892	elif l[3] is not None: # atom parameter,
1893	lbl = ShortPhaseNames.get(l[0],'phase?')
1894	try:
1895	albl = LookupAtomLabel(l[0],l[3])[0]
1896	except KeyError:
1897	albl = 'Atom?'
1898	s = "Atom "+str(albl)+" in "+str(lbl)
1899	elif l[0] == '*':
1900	s = "All phases "
1901	elif l[0] is not None:
1902	lbl = ShortPhaseNames.get(l[0],'phase?')
1903	s = "Phase "+str(lbl)
1904	elif l[1] == '*':
1905	s = 'All hists'
1906	elif l[1] is not None:
1907	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1908	if hlbl[:4] == 'HKLF':
1909	hlbl = 'Xtl='+hlbl[5:]
1910	elif hlbl[:4] == 'PWDR':
1911	hlbl = 'Pwd='+hlbl[5:]
1912	else:
1913	hlbl = 'Hist='+hlbl
1914	s = str(hlbl)
1915	if not s:
1916	s = 'Global'
1917	return s,l[-1]
1918
1919	def getVarDescr(varname):
1920	'''Return a short description for a GSAS-II variable
1921
1922	:param str name: A full G2 variable name with 2 or 3 or 4
1923	colons (<p>:<h>:name[:<a1>][:<a2>])
1924
1925	:returns: a six element list as [`p`,`h`,`name`,`a1`,`a2`,`description`],
1926	where `p`, `h`, `a1`, `a2` are str values or `None`, for the phase number,
1927	the histogram number and the atom number; `name` will always be
1928	a str; and `description` is str or `None`.
1929	If the variable name is incorrectly formed (for example, wrong
1930	number of colons), `None` is returned instead of a list.
1931	'''
1932	l = varname.split(':')
1933	if len(l) == 2: #SASD parameter name
1934	return varname,l[0],getDescr(l[1])
1935	if len(l) == 3:
1936	l += [None,None]
1937	elif len(l) == 4:
1938	l += [None]
1939	elif len(l) != 5:
1940	return None
1941	for i in (0,1,3,4):
1942	if l[i] == "":
1943	l[i] = None
1944	l += [getDescr(l[2])]
1945	return l
1946
1947	def CompileVarDesc():
1948	'''Set the values in the variable lookup tables
1949	(:attr:`reVarDesc` and :attr:`reVarStep`).
1950	This is called in :func:`getDescr` and :func:`getVarStep` so this
1951	initialization is always done before use.
1952
1953	Note that keys may contain regular expressions, where '[xyz]'
1954	matches 'x' 'y' or 'z' (equivalently '[x-z]' describes this as range
1955	of values). '.*' matches any string. For example::
1956
1957	'AUiso':'Atomic isotropic displacement parameter',
1958
1959	will match variable ``'p::AUiso:a'``.
1960	If parentheses are used in the key, the contents of those parentheses can be
1961	used in the value, such as::
1962
1963	'AU([123][123])':'Atomic anisotropic displacement parameter U\\1',
1964
1965	will match ``AU11``, ``AU23``,... and `U11`, `U23` etc will be displayed
1966	in the value when used.
1967
1968	'''
1969	if reVarDesc: return # already done
1970	for key,value in {
1971	# derived or other sequential vars
1972	'([abc])$' : 'Lattice parameter, \\1, from Ai and Djk', # N.B. '$' prevents match if any characters follow
1973	u'\u03B1' : u'Lattice parameter, \u03B1, computed from both Ai and Djk',
1974	u'\u03B2' : u'Lattice parameter, \u03B2, computed from both Ai and Djk',
1975	u'\u03B3' : u'Lattice parameter, \u03B3, computed from both Ai and Djk',
1976	# ambiguous, alas:
1977	'Scale' : 'Phase fraction (as p:h:Scale) or Histogram scale factor (as :h:Scale)',
1978	# Phase vars (p::<var>)
1979	'A([0-5])' : ('Reciprocal metric tensor component \\1',1e-5),
1980	'([vV]ol)' : 'Unit cell volume', # probably an error that both upper and lower case are used
1981	# Atom vars (p::<var>:a)
1982	'dA([xyz])$' : ('Refined change to atomic coordinate, \\1',1e-6),
1983	'A([xyz])$' : 'Fractional atomic coordinate, \\1',
1984	'AUiso':('Atomic isotropic displacement parameter',1e-4),
1985	'AU([123][123])':('Atomic anisotropic displacement parameter U\\1',1e-4),
1986	'Afrac': ('Atomic site fraction parameter',1e-5),
1987	'Amul': 'Atomic site multiplicity value',
1988	'AM([xyz])$' : 'Atomic magnetic moment parameter, \\1',
1989	# Hist (:h:<var>) & Phase (HAP) vars (p:h:<var>)
1990	'Back(.*)': 'Background term #\\1',
1991	'BkPkint;(.*)':'Background peak #\\1 intensity',
1992	'BkPkpos;(.*)':'Background peak #\\1 position',
1993	'BkPksig;(.*)':'Background peak #\\1 Gaussian width',
1994	'BkPkgam;(.*)':'Background peak #\\1 Cauchy width',
1995	# 'Back File' : 'Background file name',
1996	'BF mult' : 'Background file multiplier',
1997	'Bab([AU])': 'Babinet solvent scattering coef. \\1',
1998	'D([123][123])' : 'Anisotropic strain coef. \\1',
1999	'Extinction' : 'Extinction coef.',
2000	'MD' : 'March-Dollase coef.',
2001	'Mustrain;.' : 'Microstrain coefficient (delta Q/Q x 10*6)',
2002	'Size;.*' : 'Crystallite size value (in microns)',
2003	'eA$' : 'Cubic mustrain value',
2004	'Ep$' : 'Primary extinction',
2005	'Es$' : 'Secondary type II extinction',
2006	'Eg$' : 'Secondary type I extinction',
2007	'Flack' : 'Flack parameter',
2008	'TwinFr' : 'Twin fraction',
2009	'Layer Disp' : 'Layer displacement along beam',
2010	#Histogram vars (:h:<var>)
2011	'Absorption' : 'Absorption coef.',
2012	'LayerDisp' : 'Bragg-Brentano Layer displacement',
2013	'Displace([XY])' : ('Debye-Scherrer sample displacement \\1',0.1),
2014	'Lam' : ('Wavelength',1e-6),
2015	'I\\(L2\\)\\/I\\(L1\\)' : ('Ka2/Ka1 intensity ratio',0.001),
2016	'Polariz.' : ('Polarization correction',1e-3),
2017	'SH/L' : ('FCJ peak asymmetry correction',1e-4),
2018	'([UVW])$' : ('Gaussian instrument broadening \\1',1e-5),
2019	'([XYZ])$' : ('Cauchy instrument broadening \\1',1e-5),
2020	'Zero' : 'Debye-Scherrer zero correction',
2021	'Shift' : 'Bragg-Brentano sample displ.',
2022	'SurfRoughA' : 'Bragg-Brenano surface roughness A',
2023	'SurfRoughB' : 'Bragg-Brenano surface roughness B',
2024	'Transparency' : 'Bragg-Brentano sample tranparency',
2025	'DebyeA' : 'Debye model amplitude',
2026	'DebyeR' : 'Debye model radius',
2027	'DebyeU' : 'Debye model Uiso',
2028	'RBV.*' : 'Vector rigid body parameter',
2029	'RBVO([aijk])' : 'Vector rigid body orientation parameter \\1',
2030	'RBVP([xyz])' : 'Vector rigid body \\1 position parameter',
2031	'RBVf' : 'Vector rigid body site fraction',
2032	'RBV([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
2033	'RBVU' : 'Residue rigid body group Uiso param.',
2034	'RBRO([aijk])' : 'Residue rigid body orientation parameter \\1',
2035	'RBRP([xyz])' : 'Residue rigid body \\1 position parameter',
2036	'RBRTr;.*' : 'Residue rigid body torsion parameter',
2037	'RBRf' : 'Residue rigid body site fraction',
2038	'RBR([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
2039	'RBRU' : 'Residue rigid body group Uiso param.',
2040	'constr([0-9]*)' : 'Generated degree of freedom from constraint',
2041	'nv-(.+)' : 'New variable assignment with name \\1',
2042	# supersymmetry parameters p::<var>:a:o 'Flen','Fcent'?
2043	'mV([0-2])$' : 'Modulation vector component \\1',
2044	'Fsin' : 'Sin site fraction modulation',
2045	'Fcos' : 'Cos site fraction modulation',
2046	'Fzero' : 'Crenel function offset', #may go away
2047	'Fwid' : 'Crenel function width',
2048	'Tmin' : 'ZigZag/Block min location',
2049	'Tmax' : 'ZigZag/Block max location',
2050	'([XYZ])max': 'ZigZag/Block max value for \\1',
2051	'([XYZ])sin' : 'Sin position wave for \\1',
2052	'([XYZ])cos' : 'Cos position wave for \\1',
2053	'U([123][123])sin$' : 'Sin thermal wave for U\\1',
2054	'U([123][123])cos$' : 'Cos thermal wave for U\\1',
2055	'M([XYZ])sin$' : 'Sin mag. moment wave for \\1',
2056	'M([XYZ])cos$' : 'Cos mag. moment wave for \\1',
2057	# PDF peak parms (l:<var>;l = peak no.)
2058	'PDFpos' : 'PDF peak position',
2059	'PDFmag' : 'PDF peak magnitude',
2060	'PDFsig' : 'PDF peak std. dev.',
2061	# SASD vars (l:<var>;l = component)
2062	'Aspect ratio' : 'Particle aspect ratio',
2063	'Length' : 'Cylinder length',
2064	'Diameter' : 'Cylinder/disk diameter',
2065	'Thickness' : 'Disk thickness',
2066	'Shell thickness' : 'Multiplier to get inner(<1) or outer(>1) sphere radius',
2067	'Dist' : 'Interparticle distance',
2068	'VolFr' : 'Dense scatterer volume fraction',
2069	'epis' : 'Sticky sphere epsilon',
2070	'Sticky' : 'Stickyness',
2071	'Depth' : 'Well depth',
2072	'Width' : 'Well width',
2073	'Volume' : 'Particle volume',
2074	'Radius' : 'Sphere/cylinder/disk radius',
2075	'Mean' : 'Particle mean radius',
2076	'StdDev' : 'Standard deviation in Mean',
2077	'G$': 'Guinier prefactor',
2078	'Rg$': 'Guinier radius of gyration',
2079	'B$': 'Porod prefactor',
2080	'P$': 'Porod power',
2081	'Cutoff': 'Porod cutoff',
2082	'PkInt': 'Bragg peak intensity',
2083	'PkPos': 'Bragg peak position',
2084	'PkSig': 'Bragg peak sigma',
2085	'PkGam': 'Bragg peak gamma',
2086	'e([12][12])' : 'strain tensor e\\1', # strain vars e11, e22, e12
2087	'Dcalc': 'Calc. d-spacing',
2088	'Back$': 'background parameter',
2089	'pos$': 'peak position',
2090	'int$': 'peak intensity',
2091	'WgtFrac':'phase weight fraction',
2092	'alpha':'TOF profile term',
2093	'alpha-([01])':'Pink profile term',
2094	'beta-([01q])':'TOF/Pink profile term',
2095	'sig-([012q])':'TOF profile term',
2096	'dif([ABC])':'TOF to d-space calibration',
2097	'C\\([0-9],[0-9]\\)' : 'spherical harmonics preferred orientation coef.',
2098	'Pressure': 'Pressure level for measurement in MPa',
2099	'Temperature': 'T value for measurement, K',
2100	'FreePrm([123])': 'User defined measurement parameter \\1',
2101	'Gonio. radius': 'Distance from sample to detector, mm',
2102	}.items():
2103	# Needs documentation: HAP: LeBail, newLeBail
2104	# hist: Azimuth, Chi, Omega, Phi, Bank, nDebye, nPeaks
2105
2106	if len(value) == 2:
2107	#VarDesc[key] = value[0]
2108	reVarDesc[re.compile(key)] = value[0]
2109	reVarStep[re.compile(key)] = value[1]
2110	else:
2111	#VarDesc[key] = value
2112	reVarDesc[re.compile(key)] = value
2113
2114	def removeNonRefined(parmList):
2115	'''Remove items from variable list that are not refined and should not
2116	appear as options for constraints
2117
2118	:param list parmList: a list of strings of form "p:h:VAR:a" where
2119	VAR is the variable name
2120
2121	:returns: a list after removing variables where VAR matches a
2122	entry in local variable NonRefinedList
2123	'''
2124	NonRefinedList = ['Omega','Type','Chi','Phi', 'Azimuth','Gonio. radius',
2125	'Lam1','Lam2','Back','Temperature','Pressure',
2126	'FreePrm1','FreePrm2','FreePrm3',
2127	'Source','nPeaks','LeBail','newLeBail','Bank',
2128	'nDebye', #'',
2129	]
2130	return [prm for prm in parmList if prm.split(':')[2] not in NonRefinedList]
2131
2132	def getDescr(name):
2133	'''Return a short description for a GSAS-II variable
2134
2135	:param str name: The descriptive part of the variable name without colons (:)
2136
2137	:returns: a short description or None if not found
2138	'''
2139
2140	CompileVarDesc() # compile the regular expressions, if needed
2141	for key in reVarDesc:
2142	m = key.match(name)
2143	if m:
2144	reVarDesc[key]
2145	return m.expand(reVarDesc[key])
2146	return None
2147
2148	def getVarStep(name,parmDict=None):
2149	'''Return a step size for computing the derivative of a GSAS-II variable
2150
2151	:param str name: A complete variable name (with colons, :)
2152	:param dict parmDict: A dict with parameter values or None (default)
2153
2154	:returns: a float that should be an appropriate step size, either from
2155	the value supplied in :func:`CompileVarDesc` or based on the value for
2156	name in parmDict, if supplied. If not found or the value is zero,
2157	a default value of 1e-5 is used. If parmDict is None (default) and
2158	no value is provided in :func:`CompileVarDesc`, then None is returned.
2159	'''
2160	CompileVarDesc() # compile the regular expressions, if needed
2161	for key in reVarStep:
2162	m = key.match(name)
2163	if m:
2164	return reVarStep[key]
2165	if parmDict is None: return None
2166	val = parmDict.get(key,0.0)
2167	if abs(val) > 0.05:
2168	return abs(val)/1000.
2169	else:
2170	return 1e-5
2171
2172	def GenWildCard(varlist):
2173	'''Generate wildcard versions of G2 variables. These introduce '*'
2174	for a phase, histogram or atom number (but only for one of these
2175	fields) but only when there is more than one matching variable in the
2176	input variable list. So if the input is this::
2177
2178	varlist = ['0::AUiso:0', '0::AUiso:1', '1::AUiso:0']
2179
2180	then the output will be this::
2181
2182	wildList = ['::AUiso:0', '0::AUiso:']
2183
2184	:param list varlist: an input list of GSAS-II variable names
2185	(such as 0::AUiso:0)
2186
2187	:returns: wildList, the generated list of wild card variable names.
2188	'''
2189	wild = []
2190	for i in (0,1,3):
2191	currentL = varlist[:]
2192	while currentL:
2193	item1 = currentL.pop(0)
2194	i1splt = item1.split(':')
2195	if i >= len(i1splt): continue
2196	if i1splt[i]:
2197	nextL = []
2198	i1splt[i] = '[0-9]+'
2199	rexp = re.compile(':'.join(i1splt))
2200	matchlist = [item1]
2201	for nxtitem in currentL:
2202	if rexp.match(nxtitem):
2203	matchlist += [nxtitem]
2204	else:
2205	nextL.append(nxtitem)
2206	if len(matchlist) > 1:
2207	i1splt[i] = '*'
2208	wild.append(':'.join(i1splt))
2209	currentL = nextL
2210	return wild
2211
2212	def LookupWildCard(varname,varlist):
2213	'''returns a list of variable names from list varname
2214	that match wildcard name in varname
2215
2216	:param str varname: a G2 variable name containing a wildcard
2217	(such as \\*::var)
2218	:param list varlist: the list of all variable names used in
2219	the current project
2220	:returns: a list of matching GSAS-II variables (may be empty)
2221	'''
2222	rexp = re.compile(varname.replace('*','[0-9]+'))
2223	return sorted([var for var in varlist if rexp.match(var)])
2224
2225	def prmLookup(name,prmDict):
2226	'''Looks for a parameter in a min/max dictionary, optionally
2227	considering a wild card for histogram or atom number (use of
2228	both will never occur at the same time).
2229
2230	:param name: a GSAS-II parameter name (str, see :func:`getVarDescr`
2231	and :func:`CompileVarDesc`) or a :class:`G2VarObj` object.
2232	:param dict prmDict: a min/max dictionary, (parmMinDict
2233	or parmMaxDict in Controls) where keys are :class:`G2VarObj`
2234	objects.
2235	:returns: Two values, (matchname, value), are returned where:
2236
2237	* matchname (str) is the :class:`G2VarObj` object
2238	corresponding to the actual matched name,
2239	which could contain a wildcard even if name does not; and
2240	* value (float) which contains the parameter limit.
2241	'''
2242	for key,value in prmDict.items():
2243	if str(key) == str(name): return key,value
2244	if key == name: return key,value
2245	return None,None
2246
2247
2248	def _lookup(dic,key):
2249	'''Lookup a key in a dictionary, where None returns an empty string
2250	but an unmatched key returns a question mark. Used in :class:`G2VarObj`
2251	'''
2252	if key is None:
2253	return ""
2254	elif key == "*":
2255	return "*"
2256	else:
2257	return dic.get(key,'?')
2258
2259	def SortVariables(varlist):
2260	'''Sorts variable names in a sensible manner
2261	'''
2262	def cvnnums(var):
2263	v = []
2264	for i in var.split(':'):
2265	# if i == '' or i == '*':
2266	# v.append(-1)
2267	# continue
2268	try:
2269	v.append(int(i))
2270	except:
2271	v.append(-1)
2272	return v
2273	return sorted(varlist,key=cvnnums)
2274
2275	class G2VarObj(object):
2276	'''Defines a GSAS-II variable either using the phase/atom/histogram
2277	unique Id numbers or using a character string that specifies
2278	variables by phase/atom/histogram number (which can change).
2279	Note that :func:`GSASIIstrIO.GetUsedHistogramsAndPhases`,
2280	which calls :func:`IndexAllIds` (or
2281	:func:`GSASIIscriptable.G2Project.index_ids`) should be used to
2282	(re)load the current Ids
2283	before creating or later using the G2VarObj object.
2284
2285	This can store rigid body variables, but does not translate the residue # and
2286	body # to/from random Ids
2287
2288	A :class:`G2VarObj` object can be created with a single parameter:
2289
2290	:param str/tuple varname: a single value can be used to create a :class:`G2VarObj`
2291	object. If a string, it must be of form "p:h:var" or "p:h:var:a", where
2292
2293	* p is the phase number (which may be left blank or may be '*' to indicate all phases);
2294	* h is the histogram number (which may be left blank or may be '*' to indicate all histograms);
2295	* a is the atom number (which may be left blank in which case the third colon is omitted).
2296	The atom number can be specified as '' if a phase number is specified (not as '').
2297	For rigid body variables, specify a will be a string of form "residue:body#"
2298
2299	Alternately a single tuple of form (Phase,Histogram,VarName,AtomID) can be used, where
2300	Phase, Histogram, and AtomID are None or are ranId values (or one can be '*')
2301	and VarName is a string. Note that if Phase is '*' then the AtomID is an atom number.
2302	For a rigid body variables, AtomID is a string of form "residue:body#".
2303
2304	If four positional arguments are supplied, they are:
2305
2306	:param str/int phasenum: The number for the phase (or None or '*')
2307	:param str/int histnum: The number for the histogram (or None or '*')
2308	:param str varname: a single value can be used to create a :class:`G2VarObj`
2309	:param str/int atomnum: The number for the atom (or None or '*')
2310
2311	'''
2312	IDdict = {}
2313	IDdict['phases'] = {}
2314	IDdict['hists'] = {}
2315	IDdict['atoms'] = {}
2316	def __init__(self,*args):
2317	self.phase = None
2318	self.histogram = None
2319	self.name = ''
2320	self.atom = None
2321	if len(args) == 1 and (type(args[0]) is list or type(args[0]) is tuple) and len(args[0]) == 4:
2322	# single arg with 4 values
2323	self.phase,self.histogram,self.name,self.atom = args[0]
2324	elif len(args) == 1 and ':' in args[0]:
2325	#parse a string
2326	lst = args[0].split(':')
2327	if lst[0] == '*':
2328	self.phase = '*'
2329	if len(lst) > 3:
2330	self.atom = lst[3]
2331	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
2332	elif lst[1] == '*':
2333	self.histogram = '*'
2334	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
2335	else:
2336	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
2337	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
2338	if len(lst) == 4:
2339	if lst[3] == '*':
2340	self.atom = '*'
2341	else:
2342	self.atom = AtomIdLookup[lst[0]].get(lst[3],[None,None])[1]
2343	elif len(lst) == 5:
2344	self.atom = lst[3]+":"+lst[4]
2345	elif len(lst) == 3:
2346	pass
2347	else:
2348	raise Exception("Incorrect number of colons in var name "+str(args[0]))
2349	self.name = lst[2]
2350	elif len(args) == 4:
2351	if args[0] == '*':
2352	self.phase = '*'
2353	self.atom = args[3]
2354	else:
2355	self.phase = PhaseIdLookup.get(str(args[0]),[None,None])[1]
2356	if args[3] == '*':
2357	self.atom = '*'
2358	elif args[0] is not None:
2359	self.atom = AtomIdLookup[args[0]].get(str(args[3]),[None,None])[1]
2360	if args[1] == '*':
2361	self.histogram = '*'
2362	else:
2363	self.histogram = HistIdLookup.get(str(args[1]),[None,None])[1]
2364	self.name = args[2]
2365	else:
2366	raise Exception("Incorrectly called GSAS-II parameter name")
2367
2368	#print "DEBUG: created ",self.phase,self.histogram,self.name,self.atom
2369
2370	def __str__(self):
2371	return self.varname()
2372
2373	def __hash__(self):
2374	'Allow G2VarObj to be a dict key by implementing hashing'
2375	return hash(self.varname())
2376
2377	def varname(self,hist=None):
2378	'''Formats the GSAS-II variable name as a "traditional" GSAS-II variable
2379	string (p:h:<var>:a) or (p:h:<var>)
2380
2381	:param str/int hist: if specified, overrides the histogram number
2382	with the specified value
2383	:returns: the variable name as a str
2384	'''
2385	a = ""
2386	if self.phase == "*":
2387	ph = "*"
2388	if self.atom:
2389	a = ":" + str(self.atom)
2390	else:
2391	ph = _lookup(PhaseRanIdLookup,self.phase)
2392	if self.atom == '*':
2393	a = ':*'
2394	elif self.atom:
2395	if ":" in str(self.atom):
2396	a = ":" + str(self.atom)
2397	elif ph in AtomRanIdLookup:
2398	a = ":" + AtomRanIdLookup[ph].get(self.atom,'?')
2399	else:
2400	a = ":?"
2401	if hist is not None and self.histogram:
2402	hist = str(hist)
2403	elif self.histogram == "*":
2404	hist = "*"
2405	else:
2406	hist = _lookup(HistRanIdLookup,self.histogram)
2407	s = (ph + ":" + hist + ":" + str(self.name)) + a
2408	return s
2409
2410	def __repr__(self):
2411	'''Return the detailed contents of the object
2412	'''
2413	s = "<"
2414	if self.phase == '*':
2415	s += "Phases: all; "
2416	if self.atom is not None:
2417	if ":" in str(self.atom):
2418	s += "Rigid body" + str(self.atom) + "; "
2419	else:
2420	s += "Atom #" + str(self.atom) + "; "
2421	elif self.phase is not None:
2422	ph = _lookup(PhaseRanIdLookup,self.phase)
2423	s += "Phase: rId=" + str(self.phase) + " (#"+ ph + "); "
2424	if self.atom == '*':
2425	s += "Atoms: all; "
2426	elif ":" in str(self.atom):
2427	s += "Rigid body" + str(self.atom) + "; "
2428	elif self.atom is not None:
2429	s += "Atom rId=" + str(self.atom)
2430	if ph in AtomRanIdLookup:
2431	s += " (#" + AtomRanIdLookup[ph].get(self.atom,'?') + "); "
2432	else:
2433	s += " (#? -- not found!); "
2434	if self.histogram == '*':
2435	s += "Histograms: all; "
2436	elif self.histogram is not None:
2437	hist = _lookup(HistRanIdLookup,self.histogram)
2438	s += "Histogram: rId=" + str(self.histogram) + " (#"+ hist + "); "
2439	s += 'Variable name="' + str(self.name) + '">'
2440	return s+" ("+self.varname()+")"
2441
2442	def __eq__(self, other):
2443	'''Allow comparison of G2VarObj to other G2VarObj objects or strings.
2444	If any field is a wildcard ('*') that field matches.
2445	'''
2446	if type(other) is str:
2447	other = G2VarObj(other)
2448	elif type(other) is not G2VarObj:
2449	raise Exception("Invalid type ({}) for G2VarObj comparison with {}"
2450	.format(type(other),other))
2451	if self.phase != other.phase and self.phase != '' and other.phase != '':
2452	return False
2453	if self.histogram != other.histogram and self.histogram != '' and other.histogram != '':
2454	return False
2455	if self.atom != other.atom and self.atom != '' and other.atom != '':
2456	return False
2457	if self.name != other.name:
2458	return False
2459	return True
2460
2461	def fmtVarByMode(self, seqmode, note, warnmsg):
2462	'''Format a parameter object for display. Note that these changes
2463	are only temporary and are only shown only when the Constraints
2464	data tree is selected.
2465
2466	* In a non-sequential refinement or where the mode is 'use-all', the
2467	name is converted unchanged to a str
2468	* In a sequential refinement when the mode is 'wildcards-only' the
2469	name is converted unchanged to a str but a warning is added
2470	for non-wildcarded HAP or Histogram parameters
2471	* In a sequential refinement or where the mode is 'auto-wildcard',
2472	a histogram number is converted to a wildcard (*) and then
2473	converted to str
2474
2475	:param str mode: the sequential mode (see above)
2476	:param str note: value displayed on the line of the constraint/equiv.
2477	:param str warnmsg: a message saying the constraint is not used
2478
2479	:returns: varname, explain, note, warnmsg (all str values) where:
2480
2481	* varname is the parameter expressed as a string,
2482	* explain is blank unless there is a warning explanation about
2483	the parameter or blank
2484	* note is the previous value unless overridden
2485	* warnmsg is the previous value unless overridden
2486	'''
2487	explain = ''
2488	s = self.varname()
2489	if seqmode == 'auto-wildcard':
2490	if self.histogram: s = self.varname('*')
2491	elif seqmode == 'wildcards-only' and self.histogram:
2492	if self.histogram != '*':
2493	warnmsg = 'Ignored due to use of a non-wildcarded histogram number'
2494	note = 'Ignored'
2495	explain = '\nIgnoring: '+self.varname()+' does not contain a wildcard.\n'
2496	elif seqmode != 'use-all' and seqmode != 'wildcards-only':
2497	print('Unexpected mode',seqmode,' in fmtVarByMode')
2498	return s,explain,note,warnmsg
2499
2500	def _show(self):
2501	'For testing, shows the current lookup table'
2502	print ('phases'+ self.IDdict['phases'])
2503	print ('hists'+ self.IDdict['hists'])
2504	print ('atomDict'+ self.IDdict['atoms'])
2505
2506	#==========================================================================
2507	def SetDefaultSample():
2508	'Fills in default items for the Sample dictionary for Debye-Scherrer & SASD'
2509	return {
2510	'InstrName':'',
2511	'ranId':ran.randint(0,sys.maxsize),
2512	'Scale':[1.0,True],'Type':'Debye-Scherrer','Absorption':[0.0,False],
2513	'DisplaceX':[0.0,False],'DisplaceY':[0.0,False],
2514	'Temperature':300.,'Pressure':0.1,'Time':0.0,
2515	'FreePrm1':0.,'FreePrm2':0.,'FreePrm3':0.,
2516	'Gonio. radius':200.0,
2517	'Omega':0.0,'Chi':0.0,'Phi':0.0,'Azimuth':0.0,
2518	#SASD items
2519	'Materials':[{'Name':'vacuum','VolFrac':1.0,},{'Name':'vacuum','VolFrac':0.0,}],
2520	'Thick':1.0,'Contrast':[0.0,0.0], #contrast & anomalous contrast
2521	'Trans':1.0, #measured transmission
2522	'SlitLen':0.0, #Slit length - in Q(A-1)
2523	}
2524	######################################################################
2525	class ImportBaseclass(object):
2526	'''Defines a base class for the reading of input files (diffraction
2527	data, coordinates,...). See :ref:`Writing a Import Routine<import_routines>`
2528	for an explanation on how to use a subclass of this class.
2529	'''
2530	class ImportException(Exception):
2531	'''Defines an Exception that is used when an import routine hits an expected error,
2532	usually in .Reader.
2533
2534	Good practice is that the Reader should define a value in self.errors that
2535	tells the user some information about what is wrong with their file.
2536	'''
2537	pass
2538
2539	UseReader = True # in __init__ set value of self.UseReader to False to skip use of current importer
2540	def __init__(self,formatName,longFormatName=None,
2541	extensionlist=[],strictExtension=False,):
2542	self.formatName = formatName # short string naming file type
2543	if longFormatName: # longer string naming file type
2544	self.longFormatName = longFormatName
2545	else:
2546	self.longFormatName = formatName
2547	# define extensions that are allowed for the file type
2548	# for windows, remove any extensions that are duplicate, as case is ignored
2549	if sys.platform == 'windows' and extensionlist:
2550	extensionlist = list(set([s.lower() for s in extensionlist]))
2551	self.extensionlist = extensionlist
2552	# If strictExtension is True, the file will not be read, unless
2553	# the extension matches one in the extensionlist
2554	self.strictExtension = strictExtension
2555	self.errors = ''
2556	self.warnings = ''
2557	self.SciPy = False #image reader needed scipy
2558	# used for readers that will use multiple passes to read
2559	# more than one data block
2560	self.repeat = False
2561	self.selections = []
2562	self.repeatcount = 0
2563	self.readfilename = '?'
2564	self.scriptable = False
2565	#print 'created',self.__class__
2566
2567	def ReInitialize(self):
2568	'Reinitialize the Reader to initial settings'
2569	self.errors = ''
2570	self.warnings = ''
2571	self.SciPy = False #image reader needed scipy
2572	self.repeat = False
2573	self.repeatcount = 0
2574	self.readfilename = '?'
2575
2576
2577	# def Reader(self, filename, filepointer, ParentFrame=None, **unused):
2578	# '''This method must be supplied in the child class to read the file.
2579	# if the read fails either return False or raise an Exception
2580	# preferably of type ImportException.
2581	# '''
2582	# #start reading
2583	# raise ImportException("Error occurred while...")
2584	# self.errors += "Hint for user on why the error occur
2585	# return False # if an error occurs
2586	# return True # if read OK
2587
2588	def ExtensionValidator(self, filename):
2589	'''This methods checks if the file has the correct extension
2590
2591	:returns:
2592
2593	* False if this filename will not be supported by this reader (only
2594	when strictExtension is True)
2595	* True if the extension matches the list supplied by the reader
2596	* None if the reader allows un-registered extensions
2597
2598	'''
2599	if filename:
2600	ext = os.path.splitext(filename)[1]
2601	if not ext and self.strictExtension: return False
2602	for ext in self.extensionlist:
2603	if sys.platform == 'windows':
2604	if filename.lower().endswith(ext): return True
2605	else:
2606	if filename.endswith(ext): return True
2607	if self.strictExtension:
2608	return False
2609	else:
2610	return None
2611
2612	def ContentsValidator(self, filename):
2613	'''This routine will attempt to determine if the file can be read
2614	with the current format.
2615	This will typically be overridden with a method that
2616	takes a quick scan of [some of]
2617	the file contents to do a "sanity" check if the file
2618	appears to match the selected format.
2619	the file must be opened here with the correct format (binary/text)
2620	'''
2621	#filepointer.seek(0) # rewind the file pointer
2622	return True
2623
2624	def CIFValidator(self, filepointer):
2625	'''A :meth:`ContentsValidator` for use to validate CIF files.
2626	'''
2627	filepointer.seek(0)
2628	for i,l in enumerate(filepointer):
2629	if i >= 1000: return True
2630	'''Encountered only blank lines or comments in first 1000
2631	lines. This is unlikely, but assume it is CIF anyway, since we are
2632	even less likely to find a file with nothing but hashes and
2633	blank lines'''
2634	line = l.strip()
2635	if len(line) == 0: # ignore blank lines
2636	continue
2637	elif line.startswith('#'): # ignore comments
2638	continue
2639	elif line.startswith('data_'): # on the right track, accept this file
2640	return True
2641	else: # found something invalid
2642	self.errors = 'line '+str(i+1)+' contains unexpected data:\n'
2643	if all([ord(c) < 128 and ord(c) != 0 for c in str(l)]): # show only if ASCII
2644	self.errors += ' '+str(l)
2645	else:
2646	self.errors += ' (binary)'
2647	self.errors += '\n Note: a CIF should only have blank lines or comments before'
2648	self.errors += '\n a data_ statement begins a block.'
2649	return False
2650
2651	######################################################################
2652	class ImportPhase(ImportBaseclass):
2653	'''Defines a base class for the reading of files with coordinates
2654
2655	Objects constructed that subclass this (in import/G2phase_*.py etc.) will be used
2656	in :meth:`GSASIIdataGUI.GSASII.OnImportPhase` and in
2657	:func:`GSASIIscriptable.import_generic`.
2658	See :ref:`Writing a Import Routine<import_routines>`
2659	for an explanation on how to use this class.
2660
2661	'''
2662	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2663	strictExtension=False,):
2664	# call parent __init__
2665	ImportBaseclass.__init__(self,formatName,longFormatName,
2666	extensionlist,strictExtension)
2667	self.Phase = None # a phase must be created with G2IO.SetNewPhase in the Reader
2668	self.SymOps = {} # specified when symmetry ops are in file (e.g. CIF)
2669	self.Constraints = None
2670
2671	######################################################################
2672	class ImportStructFactor(ImportBaseclass):
2673	'''Defines a base class for the reading of files with tables
2674	of structure factors.
2675
2676	Structure factors are read with a call to :meth:`GSASIIdataGUI.GSASII.OnImportSfact`
2677	which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`, which calls
2678	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2679	:meth:`Reader`.
2680
2681	See :ref:`Writing a Import Routine<import_routines>`
2682	for an explanation on how to use import classes in general. The specifics
2683	for reading a structure factor histogram require that
2684	the ``Reader()`` routine in the import
2685	class need to do only a few things: It
2686	should load :attr:`RefDict` item ``'RefList'`` with the reflection list,
2687	and set :attr:`Parameters` with the instrument parameters
2688	(initialized with :meth:`InitParameters` and set with :meth:`UpdateParameters`).
2689	'''
2690	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2691	strictExtension=False,):
2692	ImportBaseclass.__init__(self,formatName,longFormatName,
2693	extensionlist,strictExtension)
2694
2695	# define contents of Structure Factor entry
2696	self.Parameters = []
2697	'self.Parameters is a list with two dicts for data parameter settings'
2698	self.InitParameters()
2699	self.RefDict = {'RefList':[],'FF':{},'Super':0}
2700	self.Banks = [] #for multi bank data (usually TOF)
2701	'''self.RefDict is a dict containing the reflection information, as read from the file.
2702	Item 'RefList' contains the reflection information. See the
2703	:ref:`Single Crystal Reflection Data Structure<XtalRefl_table>`
2704	for the contents of each row. Dict element 'FF'
2705	contains the form factor values for each element type; if this entry
2706	is left as initialized (an empty list) it will be initialized as needed later.
2707	'''
2708	def ReInitialize(self):
2709	'Reinitialize the Reader to initial settings'
2710	ImportBaseclass.ReInitialize(self)
2711	self.InitParameters()
2712	self.Banks = [] #for multi bank data (usually TOF)
2713	self.RefDict = {'RefList':[],'FF':{},'Super':0}
2714
2715	def InitParameters(self):
2716	'initialize the instrument parameters structure'
2717	Lambda = 0.70926
2718	HistType = 'SXC'
2719	self.Parameters = [{'Type':[HistType,HistType], # create the structure
2720	'Lam':[Lambda,Lambda]
2721	}, {}]
2722	'Parameters is a list with two dicts for data parameter settings'
2723
2724	def UpdateParameters(self,Type=None,Wave=None):
2725	'Revise the instrument parameters'
2726	if Type is not None:
2727	self.Parameters[0]['Type'] = [Type,Type]
2728	if Wave is not None:
2729	self.Parameters[0]['Lam'] = [Wave,Wave]
2730
2731	######################################################################
2732	class ImportPowderData(ImportBaseclass):
2733	'''Defines a base class for the reading of files with powder data.
2734
2735	Objects constructed that subclass this (in import/G2pwd_*.py etc.) will be used
2736	in :meth:`GSASIIdataGUI.GSASII.OnImportPowder` and in
2737	:func:`GSASIIscriptable.import_generic`.
2738	See :ref:`Writing a Import Routine<import_routines>`
2739	for an explanation on how to use this class.
2740	'''
2741	def __init__(self,formatName,longFormatName=None,
2742	extensionlist=[],strictExtension=False,):
2743	ImportBaseclass.__init__(self,formatName,longFormatName,
2744	extensionlist,strictExtension)
2745	self.clockWd = None # used in TOF
2746	self.ReInitialize()
2747
2748	def ReInitialize(self):
2749	'Reinitialize the Reader to initial settings'
2750	ImportBaseclass.ReInitialize(self)
2751	self.powderentry = ['',None,None] # (filename,Pos,Bank)
2752	self.powderdata = [] # Powder dataset
2753	'''A powder data set is a list with items [x,y,w,yc,yb,yd]:
2754	np.array(x), # x-axis values
2755	np.array(y), # powder pattern intensities
2756	np.array(w), # 1/sig(intensity)^2 values (weights)
2757	np.array(yc), # calc. intensities (zero)
2758	np.array(yb), # calc. background (zero)
2759	np.array(yd), # obs-calc profiles
2760	'''
2761	self.comments = []
2762	self.idstring = ''
2763	self.Sample = SetDefaultSample() # default sample parameters
2764	self.Controls = {} # items to be placed in top-level Controls
2765	self.GSAS = None # used in TOF
2766	self.repeat_instparm = True # Should a parm file be
2767	# used for multiple histograms?
2768	self.instparm = None # name hint from file of instparm to use
2769	self.instfile = '' # full path name to instrument parameter file
2770	self.instbank = '' # inst parm bank number
2771	self.instmsg = '' # a label that gets printed to show
2772	# where instrument parameters are from
2773	self.numbanks = 1
2774	self.instdict = {} # place items here that will be transferred to the instrument parameters
2775	self.pwdparms = {} # place parameters that are transferred directly to the tree
2776	# here (typically from an existing GPX file)
2777	######################################################################
2778	class ImportSmallAngleData(ImportBaseclass):
2779	'''Defines a base class for the reading of files with small angle data.
2780	See :ref:`Writing a Import Routine<import_routines>`
2781	for an explanation on how to use this class.
2782	'''
2783	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2784	strictExtension=False,):
2785
2786	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2787	strictExtension)
2788	self.ReInitialize()
2789
2790	def ReInitialize(self):
2791	'Reinitialize the Reader to initial settings'
2792	ImportBaseclass.ReInitialize(self)
2793	self.smallangleentry = ['',None,None] # (filename,Pos,Bank)
2794	self.smallangledata = [] # SASD dataset
2795	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2796	np.array(x), # x-axis values
2797	np.array(y), # powder pattern intensities
2798	np.array(w), # 1/sig(intensity)^2 values (weights)
2799	np.array(yc), # calc. intensities (zero)
2800	np.array(yd), # obs-calc profiles
2801	np.array(yb), # preset bkg
2802	'''
2803	self.comments = []
2804	self.idstring = ''
2805	self.Sample = SetDefaultSample()
2806	self.GSAS = None # used in TOF
2807	self.clockWd = None # used in TOF
2808	self.numbanks = 1
2809	self.instdict = {} # place items here that will be transferred to the instrument parameters
2810
2811	######################################################################
2812	class ImportReflectometryData(ImportBaseclass):
2813	'''Defines a base class for the reading of files with reflectometry data.
2814	See :ref:`Writing a Import Routine<import_routines>`
2815	for an explanation on how to use this class.
2816	'''
2817	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2818	strictExtension=False,):
2819
2820	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2821	strictExtension)
2822	self.ReInitialize()
2823
2824	def ReInitialize(self):
2825	'Reinitialize the Reader to initial settings'
2826	ImportBaseclass.ReInitialize(self)
2827	self.reflectometryentry = ['',None,None] # (filename,Pos,Bank)
2828	self.reflectometrydata = [] # SASD dataset
2829	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2830	np.array(x), # x-axis values
2831	np.array(y), # powder pattern intensities
2832	np.array(w), # 1/sig(intensity)^2 values (weights)
2833	np.array(yc), # calc. intensities (zero)
2834	np.array(yd), # obs-calc profiles
2835	np.array(yb), # preset bkg
2836	'''
2837	self.comments = []
2838	self.idstring = ''
2839	self.Sample = SetDefaultSample()
2840	self.GSAS = None # used in TOF
2841	self.clockWd = None # used in TOF
2842	self.numbanks = 1
2843	self.instdict = {} # place items here that will be transferred to the instrument parameters
2844
2845	######################################################################
2846	class ImportPDFData(ImportBaseclass):
2847	'''Defines a base class for the reading of files with PDF G(R) data.
2848	See :ref:`Writing a Import Routine<import_routines>`
2849	for an explanation on how to use this class.
2850	'''
2851	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2852	strictExtension=False,):
2853
2854	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2855	strictExtension)
2856	self.ReInitialize()
2857
2858	def ReInitialize(self):
2859	'Reinitialize the Reader to initial settings'
2860	ImportBaseclass.ReInitialize(self)
2861	self.pdfentry = ['',None,None] # (filename,Pos,Bank)
2862	self.pdfdata = [] # PDF G(R) dataset
2863	'''A pdf g(r) data set is a list with items [x,y]:
2864	np.array(x), # r-axis values
2865	np.array(y), # pdf g(r)
2866	'''
2867	self.comments = []
2868	self.idstring = ''
2869	self.numbanks = 1
2870
2871	######################################################################
2872	class ImportImage(ImportBaseclass):
2873	'''Defines a base class for the reading of images
2874
2875	Images are read in only these places:
2876
2877	* Initial reading is typically done from a menu item
2878	with a call to :meth:`GSASIIdataGUI.GSASII.OnImportImage`
2879	which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`. That calls
2880	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2881	:meth:`Reader`. This returns a list of reader objects for each read image.
2882	Also used in :func:`GSASIIscriptable.import_generic`.
2883
2884	* Images are read alternatively in :func:`GSASIIIO.ReadImages`, which puts image info
2885	directly into the data tree.
2886
2887	* Images are reloaded with :func:`GSASIIIO.GetImageData`.
2888
2889	When reading an image, the ``Reader()`` routine in the ImportImage class
2890	should set:
2891
2892	* :attr:`Comments`: a list of strings (str),
2893	* :attr:`Npix`: the number of pixels in the image (int),
2894	* :attr:`Image`: the actual image as a numpy array (np.array)
2895	* :attr:`Data`: a dict defining image parameters (dict). Within this dict the following
2896	data items are needed:
2897
2898	* 'pixelSize': size of each pixel in microns (such as ``[200.,200.]``.
2899	* 'wavelength': wavelength in :math:`\\AA`.
2900	* 'distance': distance of detector from sample in cm.
2901	* 'center': uncalibrated center of beam on detector (such as ``[204.8,204.8]``.
2902	* 'size': size of image (such as ``[2048,2048]``).
2903	* 'ImageTag': image number or other keyword used to retrieve image from
2904	a multi-image data file (defaults to ``1`` if not specified).
2905	* 'sumfile': holds sum image file name if a sum was produced from a multi image file
2906
2907	optional data items:
2908
2909	* :attr:`repeat`: set to True if there are additional images to
2910	read in the file, False otherwise
2911	* :attr:`repeatcount`: set to the number of the image.
2912
2913	Note that the above is initialized with :meth:`InitParameters`.
2914	(Also see :ref:`Writing a Import Routine<import_routines>`
2915	for an explanation on how to use import classes in general.)
2916	'''
2917	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2918	strictExtension=False,):
2919	ImportBaseclass.__init__(self,formatName,longFormatName,
2920	extensionlist,strictExtension)
2921	self.InitParameters()
2922
2923	def ReInitialize(self):
2924	'Reinitialize the Reader to initial settings -- not used at present'
2925	ImportBaseclass.ReInitialize(self)
2926	self.InitParameters()
2927
2928	def InitParameters(self):
2929	'initialize the instrument parameters structure'
2930	self.Comments = ['No comments']
2931	self.Data = {'samplechangerpos':0.0,'det2theta':0.0,'Gain map':''}
2932	self.Npix = 0
2933	self.Image = None
2934	self.repeat = False
2935	self.repeatcount = 1
2936	self.sumfile = ''
2937
2938	def LoadImage(self,ParentFrame,imagefile,imagetag=None):
2939	'''Optionally, call this after reading in an image to load it into the tree.
2940	This saves time by preventing a reread of the same information.
2941	'''
2942	if ParentFrame:
2943	ParentFrame.ImageZ = self.Image # store the image for plotting
2944	ParentFrame.oldImagefile = imagefile # save the name of the last image file read
2945	ParentFrame.oldImageTag = imagetag # save the tag of the last image file read
2946
2947	#################################################################################################
2948	# shortcut routines
2949	exp = np.exp
2950	sind = sin = s = lambda x: np.sin(x*np.pi/180.)
2951	cosd = cos = c = lambda x: np.cos(x*np.pi/180.)
2952	tand = tan = t = lambda x: np.tan(x*np.pi/180.)
2953	sqrt = sq = lambda x: np.sqrt(x)
2954	pi = lambda: np.pi
2955
2956	def FindFunction(f):
2957	'''Find the object corresponding to function f
2958
2959	:param str f: a function name such as 'numpy.exp'
2960	:returns: (pkgdict,pkgobj) where pkgdict contains a dict
2961	that defines the package location(s) and where pkgobj
2962	defines the object associated with the function.
2963	If the function is not found, pkgobj is None.
2964	'''
2965	df = f.split('.')
2966	pkgdict = {}
2967	# no listed module name, try in current namespace
2968	if len(df) == 1:
2969	try:
2970	fxnobj = eval(f)
2971	return pkgdict,fxnobj
2972	except (AttributeError, NameError):
2973	return None,None
2974
2975	# includes a package, see if package is already imported
2976	pkgnam = '.'.join(df[:-1])
2977	try:
2978	fxnobj = eval(f)
2979	pkgdict[pkgnam] = eval(pkgnam)
2980	return pkgdict,fxnobj
2981	except (AttributeError, NameError):
2982	pass
2983	# package not yet imported, so let's try
2984	if '.' not in sys.path: sys.path.append('.')
2985	pkgnam = '.'.join(df[:-1])
2986	#for pkg in f.split('.')[:-1]: # if needed, descend down the tree
2987	# if pkgname:
2988	# pkgname += '.' + pkg
2989	# else:
2990	# pkgname = pkg
2991	try:
2992	exec('import '+pkgnam)
2993	pkgdict[pkgnam] = eval(pkgnam)
2994	fxnobj = eval(f)
2995	except Exception as msg:
2996	print('load of '+pkgnam+' failed with error='+str(msg))
2997	return {},None
2998	# can we access the function? I am not exactly sure what
2999	# I intended this to test originally (BHT)
3000	try:
3001	fxnobj = eval(f,globals(),pkgdict)
3002	return pkgdict,fxnobj
3003	except Exception as msg:
3004	print('call to',f,' failed with error=',str(msg))
3005	return None,None # not found
3006
3007	class ExpressionObj(object):
3008	'''Defines an object with a user-defined expression, to be used for
3009	secondary fits or restraints. Object is created null, but is changed
3010	using :meth:`LoadExpression`. This contains only the minimum
3011	information that needs to be stored to save and load the expression
3012	and how it is mapped to GSAS-II variables.
3013	'''
3014	def __init__(self):
3015	self.expression = ''
3016	'The expression as a text string'
3017	self.assgnVars = {}
3018	'''A dict where keys are label names in the expression mapping to a GSAS-II
3019	variable. The value a G2 variable name.
3020	Note that the G2 variable name may contain a wild-card and correspond to
3021	multiple values.
3022	'''
3023	self.freeVars = {}
3024	'''A dict where keys are label names in the expression mapping to a free
3025	parameter. The value is a list with:
3026
3027	* a name assigned to the parameter
3028	* a value for to the parameter and
3029	* a flag to determine if the variable is refined.
3030	'''
3031	self.depVar = None
3032
3033	self.lastError = ('','')
3034	'''Shows last encountered error in processing expression
3035	(list of 1-3 str values)'''
3036
3037	self.distance_dict = None # to be used for defining atom phase/symmetry info
3038	self.distance_atoms = None # to be used for defining atom distances
3039
3040	def LoadExpression(self,expr,exprVarLst,varSelect,varName,varValue,varRefflag):
3041	'''Load the expression and associated settings into the object. Raises
3042	an exception if the expression is not parsed, if not all functions
3043	are defined or if not all needed parameter labels in the expression
3044	are defined.
3045
3046	This will not test if the variable referenced in these definitions
3047	are actually in the parameter dictionary. This is checked when the
3048	computation for the expression is done in :meth:`SetupCalc`.
3049
3050	:param str expr: the expression
3051	:param list exprVarLst: parameter labels found in the expression
3052	:param dict varSelect: this will be 0 for Free parameters
3053	and non-zero for expression labels linked to G2 variables.
3054	:param dict varName: Defines a name (str) associated with each free parameter
3055	:param dict varValue: Defines a value (float) associated with each free parameter
3056	:param dict varRefflag: Defines a refinement flag (bool)
3057	associated with each free parameter
3058	'''
3059	self.expression = expr
3060	self.compiledExpr = None
3061	self.freeVars = {}
3062	self.assgnVars = {}
3063	for v in exprVarLst:
3064	if varSelect[v] == 0:
3065	self.freeVars[v] = [
3066	varName.get(v),
3067	varValue.get(v),
3068	varRefflag.get(v),
3069	]
3070	else:
3071	self.assgnVars[v] = varName[v]
3072	self.CheckVars()
3073
3074	def EditExpression(self,exprVarLst,varSelect,varName,varValue,varRefflag):
3075	'''Load the expression and associated settings from the object into
3076	arrays used for editing.
3077
3078	:param list exprVarLst: parameter labels found in the expression
3079	:param dict varSelect: this will be 0 for Free parameters
3080	and non-zero for expression labels linked to G2 variables.
3081	:param dict varName: Defines a name (str) associated with each free parameter
3082	:param dict varValue: Defines a value (float) associated with each free parameter
3083	:param dict varRefflag: Defines a refinement flag (bool)
3084	associated with each free parameter
3085
3086	:returns: the expression as a str
3087	'''
3088	for v in self.freeVars:
3089	varSelect[v] = 0
3090	varName[v] = self.freeVars[v][0]
3091	varValue[v] = self.freeVars[v][1]
3092	varRefflag[v] = self.freeVars[v][2]
3093	for v in self.assgnVars:
3094	varSelect[v] = 1
3095	varName[v] = self.assgnVars[v]
3096	return self.expression
3097
3098	def GetVaried(self):
3099	'Returns the names of the free parameters that will be refined'
3100	return ["::"+self.freeVars[v][0] for v in self.freeVars if self.freeVars[v][2]]
3101
3102	def GetVariedVarVal(self):
3103	'Returns the names and values of the free parameters that will be refined'
3104	return [("::"+self.freeVars[v][0],self.freeVars[v][1]) for v in self.freeVars if self.freeVars[v][2]]
3105
3106	def UpdateVariedVars(self,varyList,values):
3107	'Updates values for the free parameters (after a refinement); only updates refined vars'
3108	for v in self.freeVars:
3109	if not self.freeVars[v][2]: continue
3110	if "::"+self.freeVars[v][0] not in varyList: continue
3111	indx = list(varyList).index("::"+self.freeVars[v][0])
3112	self.freeVars[v][1] = values[indx]
3113
3114	def GetIndependentVars(self):
3115	'Returns the names of the required independent parameters used in expression'
3116	return [self.assgnVars[v] for v in self.assgnVars]
3117
3118	def CheckVars(self):
3119	'''Check that the expression can be parsed, all functions are
3120	defined and that input loaded into the object is internally
3121	consistent. If not an Exception is raised.
3122
3123	:returns: a dict with references to packages needed to
3124	find functions referenced in the expression.
3125	'''
3126	ret = self.ParseExpression(self.expression)
3127	if not ret:
3128	raise Exception("Expression parse error")
3129	exprLblList,fxnpkgdict = ret
3130	# check each var used in expression is defined
3131	defined = list(self.assgnVars.keys()) + list(self.freeVars.keys())
3132	notfound = []
3133	for var in exprLblList:
3134	if var not in defined:
3135	notfound.append(var)
3136	if notfound:
3137	msg = 'Not all variables defined'
3138	msg1 = 'The following variables were not defined: '
3139	msg2 = ''
3140	for var in notfound:
3141	if msg: msg += ', '
3142	msg += var
3143	self.lastError = (msg1,' '+msg2)
3144	raise Exception(msg)
3145	return fxnpkgdict
3146
3147	def ParseExpression(self,expr):
3148	'''Parse an expression and return a dict of called functions and
3149	the variables used in the expression. Returns None in case an error
3150	is encountered. If packages are referenced in functions, they are loaded
3151	and the functions are looked up into the modules global
3152	workspace.
3153
3154	Note that no changes are made to the object other than
3155	saving an error message, so that this can be used for testing prior
3156	to the save.
3157
3158	:returns: a list of used variables
3159	'''
3160	self.lastError = ('','')
3161	import ast
3162	def ASTtransverse(node,fxn=False):
3163	'''Transverse a AST-parsed expresson, compiling a list of variables
3164	referenced in the expression. This routine is used recursively.
3165
3166	:returns: varlist,fxnlist where
3167	varlist is a list of referenced variable names and
3168	fxnlist is a list of used functions
3169	'''
3170	varlist = []
3171	fxnlist = []
3172	if isinstance(node, list):
3173	for b in node:
3174	v,f = ASTtransverse(b,fxn)
3175	varlist += v
3176	fxnlist += f
3177	elif isinstance(node, ast.AST):
3178	for a, b in ast.iter_fields(node):
3179	if isinstance(b, ast.AST):
3180	if a == 'func':
3181	fxnlist += ['.'.join(ASTtransverse(b,True)[0])]
3182	continue
3183	v,f = ASTtransverse(b,fxn)
3184	varlist += v
3185	fxnlist += f
3186	elif isinstance(b, list):
3187	v,f = ASTtransverse(b,fxn)
3188	varlist += v
3189	fxnlist += f
3190	elif node.__class__.__name__ == "Name":
3191	varlist += [b]
3192	elif fxn and node.__class__.__name__ == "Attribute":
3193	varlist += [b]
3194	return varlist,fxnlist
3195	try:
3196	exprast = ast.parse(expr)
3197	except SyntaxError:
3198	s = ''
3199	import traceback
3200	for i in traceback.format_exc().splitlines()[-3:-1]:
3201	if s: s += "\n"
3202	s += str(i)
3203	self.lastError = ("Error parsing expression:",s)
3204	return
3205	# find the variables & functions
3206	v,f = ASTtransverse(exprast)
3207	varlist = sorted(list(set(v)))
3208	fxnlist = list(set(f))
3209	pkgdict = {}
3210	# check the functions are defined
3211	for fxn in fxnlist:
3212	fxndict,fxnobj = FindFunction(fxn)
3213	if not fxnobj:
3214	self.lastError = ("Error: Invalid function",fxn,
3215	"is not defined")
3216	return
3217	if not hasattr(fxnobj,'__call__'):
3218	self.lastError = ("Error: Not a function.",fxn,
3219	"cannot be called as a function")
3220	return
3221	pkgdict.update(fxndict)
3222	return varlist,pkgdict
3223
3224	def GetDepVar(self):
3225	'return the dependent variable, or None'
3226	return self.depVar
3227
3228	def SetDepVar(self,var):
3229	'Set the dependent variable, if used'
3230	self.depVar = var
3231	#==========================================================================
3232	class ExpressionCalcObj(object):
3233	'''An object used to evaluate an expression from a :class:`ExpressionObj`
3234	object.
3235
3236	:param ExpressionObj exprObj: a :class:`~ExpressionObj` expression object with
3237	an expression string and mappings for the parameter labels in that object.
3238	'''
3239	def __init__(self,exprObj):
3240	self.eObj = exprObj
3241	'The expression and mappings; a :class:`ExpressionObj` object'
3242	self.compiledExpr = None
3243	'The expression as compiled byte-code'
3244	self.exprDict = {}
3245	'''dict that defines values for labels used in expression and packages
3246	referenced by functions
3247	'''
3248	self.lblLookup = {}
3249	'''Lookup table that specifies the expression label name that is
3250	tied to a particular GSAS-II parameters in the parmDict.
3251	'''
3252	self.fxnpkgdict = {}
3253	'''a dict with references to packages needed to
3254	find functions referenced in the expression.
3255	'''
3256	self.varLookup = {}
3257	'''Lookup table that specifies the GSAS-II variable(s)
3258	indexed by the expression label name. (Used for only for diagnostics
3259	not evaluation of expression.)
3260	'''
3261	self.su = None
3262	'''Standard error evaluation where supplied by the evaluator
3263	'''
3264	# Patch: for old-style expressions with a (now removed step size)
3265	if '2' in platform.python_version_tuple()[0]:
3266	basestr = basestring
3267	else:
3268	basestr = str
3269	for v in self.eObj.assgnVars:
3270	if not isinstance(self.eObj.assgnVars[v], basestr):
3271	self.eObj.assgnVars[v] = self.eObj.assgnVars[v][0]
3272	self.parmDict = {}
3273	'''A copy of the parameter dictionary, for distance and angle computation
3274	'''
3275
3276	def SetupCalc(self,parmDict):
3277	'''Do all preparations to use the expression for computation.
3278	Adds the free parameter values to the parameter dict (parmDict).
3279	'''
3280	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
3281	return
3282	self.fxnpkgdict = self.eObj.CheckVars()
3283	# all is OK, compile the expression
3284	self.compiledExpr = compile(self.eObj.expression,'','eval')
3285
3286	# look at first value in parmDict to determine its type
3287	parmsInList = True
3288	if '2' in platform.python_version_tuple()[0]:
3289	basestr = basestring
3290	else:
3291	basestr = str
3292	for key in parmDict:
3293	val = parmDict[key]
3294	if isinstance(val, basestr):
3295	parmsInList = False
3296	break
3297	try: # check if values are in lists
3298	val = parmDict[key][0]
3299	except (TypeError,IndexError):
3300	parmsInList = False
3301	break
3302
3303	# set up the dicts needed to speed computations
3304	self.exprDict = {}
3305	self.lblLookup = {}
3306	self.varLookup = {}
3307	for v in self.eObj.freeVars:
3308	varname = self.eObj.freeVars[v][0]
3309	varname = "::" + varname.lstrip(':').replace(' ','_').replace(':',';')
3310	self.lblLookup[varname] = v
3311	self.varLookup[v] = varname
3312	if parmsInList:
3313	parmDict[varname] = [self.eObj.freeVars[v][1],self.eObj.freeVars[v][2]]
3314	else:
3315	parmDict[varname] = self.eObj.freeVars[v][1]
3316	self.exprDict[v] = self.eObj.freeVars[v][1]
3317	for v in self.eObj.assgnVars:
3318	varname = self.eObj.assgnVars[v]
3319	if varname in parmDict:
3320	self.lblLookup[varname] = v
3321	self.varLookup[v] = varname
3322	if parmsInList:
3323	self.exprDict[v] = parmDict[varname][0]
3324	else:
3325	self.exprDict[v] = parmDict[varname]
3326	elif '*' in varname:
3327	varlist = LookupWildCard(varname,list(parmDict.keys()))
3328	if len(varlist) == 0:
3329	self.exprDict[v] = None
3330	self.lblLookup[v] = varname # needed?
3331	self.exprDict.update(self.fxnpkgdict) # needed?
3332	return
3333	for var in varlist:
3334	self.lblLookup[var] = v
3335	if parmsInList:
3336	self.exprDict[v] = np.array([parmDict[var][0] for var in varlist])
3337	else:
3338	self.exprDict[v] = np.array([parmDict[var] for var in varlist])
3339	self.varLookup[v] = [var for var in varlist]
3340	else:
3341	self.exprDict[v] = None
3342	# raise Exception,"No value for variable "+str(v)
3343	self.exprDict.update(self.fxnpkgdict)
3344
3345	def UpdateVars(self,varList,valList):
3346	'''Update the dict for the expression with a set of values
3347	:param list varList: a list of variable names
3348	:param list valList: a list of corresponding values
3349	'''
3350	for var,val in zip(varList,valList):
3351	self.exprDict[self.lblLookup.get(var,'undefined: '+var)] = val
3352
3353	def UpdateDict(self,parmDict):
3354	'''Update the dict for the expression with values in a dict
3355	:param dict parmDict: a dict of values, items not in use are ignored
3356	'''
3357	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
3358	self.parmDict = parmDict
3359	return
3360	for var in parmDict:
3361	if var in self.lblLookup:
3362	self.exprDict[self.lblLookup[var]] = parmDict[var]
3363
3364	def EvalExpression(self):
3365	'''Evaluate an expression. Note that the expression
3366	and mapping are taken from the :class:`ExpressionObj` expression object
3367	and the parameter values were specified in :meth:`SetupCalc`.
3368	:returns: a single value for the expression. If parameter
3369	values are arrays (for example, from wild-carded variable names),
3370	the sum of the resulting expression is returned.
3371
3372	For example, if the expression is ``'A*B'``,
3373	where A is 2.0 and B maps to ``'1::Afrac:*'``, which evaluates to::
3374
3375	[0.5, 1, 0.5]
3376
3377	then the result will be ``4.0``.
3378	'''
3379	self.su = None
3380	if self.eObj.expression.startswith('Dist'):
3381	# GSASIIpath.IPyBreak()
3382	dist = G2mth.CalcDist(self.eObj.distance_dict, self.eObj.distance_atoms, self.parmDict)
3383	return dist
3384	elif self.eObj.expression.startswith('Angle'):
3385	angle = G2mth.CalcAngle(self.eObj.angle_dict, self.eObj.angle_atoms, self.parmDict)
3386	return angle
3387	if self.compiledExpr is None:
3388	raise Exception("EvalExpression called before SetupCalc")
3389	try:
3390	val = eval(self.compiledExpr,globals(),self.exprDict)
3391	except TypeError:
3392	val = None
3393	except NameError:
3394	val = None
3395	if not np.isscalar(val):
3396	val = np.sum(val)
3397	return val
3398
3399	def makeAngleObj(Phase,Oatom,Tatoms):
3400	General = Phase['General']
3401	cx,ct = General['AtomPtrs'][:2]
3402	pId = Phase['pId']
3403	SGData = General['SGData']
3404	Atoms = Phase['Atoms']
3405	aNames = [atom[ct-1] for atom in Atoms]
3406	tIds = []
3407	symNos = []
3408	cellNos = []
3409	oId = aNames.index(Oatom)
3410	for Tatom in Tatoms.split(';'):
3411	sB = Tatom.find('(')+1
3412	symNo = 0
3413	if sB:
3414	sF = Tatom.find(')')
3415	symNo = int(Tatom[sB:sF])
3416	symNos.append(symNo)
3417	cellNo = [0,0,0]
3418	cB = Tatom.find('[')
3419	if cB>0:
3420	cF = Tatom.find(']')+1
3421	cellNo = eval(Tatom[cB:cF])
3422	cellNos.append(cellNo)
3423	tIds.append(aNames.index(Tatom.split('+')[0]))
3424	# create an expression object
3425	obj = ExpressionObj()
3426	obj.expression = 'Angle(%s,%s,\n%s)'%(Tatoms[0],Oatom,Tatoms[1])
3427	obj.angle_dict = {'pId':pId,'SGData':SGData,'symNo':symNos,'cellNo':cellNos}
3428	obj.angle_atoms = [oId,tIds]
3429	return obj
3430
3431	class G2Exception(Exception):
3432	'A generic GSAS-II exception class'
3433	def __init__(self,msg):
3434	self.msg = msg
3435	def __str__(self):
3436	return repr(self.msg)
3437
3438	class G2RefineCancel(Exception):
3439	'Raised when Cancel is pressed in a refinement dialog'
3440	def __init__(self,msg):
3441	self.msg = msg
3442	def __str__(self):
3443	return repr(self.msg)
3444
3445	def HowDidIgetHere(wherecalledonly=False):
3446	'''Show a traceback with calls that brought us to the current location.
3447	Used for debugging.
3448	'''
3449	import traceback
3450	if wherecalledonly:
3451	i = traceback.format_list(traceback.extract_stack()[:-1])[-2]
3452	print(i.strip().rstrip())
3453	else:
3454	print (70'')
3455	for i in traceback.format_list(traceback.extract_stack()[:-1]): print(i.strip().rstrip())
3456	print (70'')
3457
3458	# Note that this is GUI code and should be moved at somepoint
3459	def CreatePDFitems(G2frame,PWDRtree,ElList,Qlimits,numAtm=1,FltBkg=0,PDFnames=[]):
3460	'''Create and initialize a new set of PDF tree entries
3461
3462	:param Frame G2frame: main GSAS-II tree frame object
3463	:param str PWDRtree: name of PWDR to be used to create PDF item
3464	:param dict ElList: data structure with composition
3465	:param list Qlimits: Q limits to be used for computing the PDF
3466	:param float numAtm: no. atom in chemical formula
3467	:param float FltBkg: flat background value
3468	:param list PDFnames: previously used PDF names
3469
3470	:returns: the Id of the newly created PDF entry
3471	'''
3472	PDFname = 'PDF '+PWDRtree[4:] # this places two spaces after PDF, which is needed is some places
3473	if PDFname in PDFnames:
3474	print('Skipping, entry already exists: '+PDFname)
3475	return None
3476	#PDFname = MakeUniqueLabel(PDFname,PDFnames)
3477	Id = G2frame.GPXtree.AppendItem(parent=G2frame.root,text=PDFname)
3478	Data = {
3479	'Sample':{'Name':PWDRtree,'Mult':1.0},
3480	'Sample Bkg.':{'Name':'','Mult':-1.0,'Refine':False},
3481	'Container':{'Name':'','Mult':-1.0,'Refine':False},
3482	'Container Bkg.':{'Name':'','Mult':-1.0},'ElList':ElList,
3483	'Geometry':'Cylinder','Diam':1.0,'Pack':0.50,'Form Vol':10.0*numAtm,'Flat Bkg':FltBkg,
3484	'DetType':'Area detector','ObliqCoeff':0.3,'Ruland':0.025,'QScaleLim':Qlimits,
3485	'Lorch':False,'BackRatio':0.0,'Rmax':100.,'noRing':False,'IofQmin':1.0,'Rmin':1.0,
3486	'I(Q)':[],'S(Q)':[],'F(Q)':[],'G(R)':[],
3487	#items for sequential PDFfit
3488	'Datarange':[0.,30.],'Fitrange':[0.,30.],'qdamp':[0.03,False],'qbroad':[0,False],'Temp':300}
3489	G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Controls'),Data)
3490	G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Peaks'),
3491	{'Limits':[1.,5.],'Background':[2,[0.,-0.2*np.pi],False],'Peaks':[]})
3492	return Id
3493
3494	class ShowTiming(object):
3495	'''An object to use for timing repeated sections of code.
3496
3497	Create the object with::
3498	tim0 = ShowTiming()
3499
3500	Tag sections of code to be timed with::
3501	tim0.start('start')
3502	tim0.start('in section 1')
3503	tim0.start('in section 2')
3504
3505	etc. (Note that each section should have a unique label.)
3506
3507	After the last section, end timing with::
3508	tim0.end()
3509
3510	Show timing results with::
3511	tim0.show()
3512
3513	'''
3514	def __init__(self):
3515	self.timeSum = []
3516	self.timeStart = []
3517	self.label = []
3518	self.prev = None
3519	def start(self,label):
3520	import time
3521	if label in self.label:
3522	i = self.label.index(label)
3523	self.timeStart[i] = time.time()
3524	else:
3525	i = len(self.label)
3526	self.timeSum.append(0.0)
3527	self.timeStart.append(time.time())
3528	self.label.append(label)
3529	if self.prev is not None:
3530	self.timeSum[self.prev] += self.timeStart[i] - self.timeStart[self.prev]
3531	self.prev = i
3532	def end(self):
3533	import time
3534	if self.prev is not None:
3535	self.timeSum[self.prev] += time.time() - self.timeStart[self.prev]
3536	self.prev = None
3537	def show(self):
3538	sumT = sum(self.timeSum)
3539	print('Timing results (total={:.2f} sec)'.format(sumT))
3540	for i,(lbl,val) in enumerate(zip(self.label,self.timeSum)):
3541	print('{} {:20} {:8.2f} ms {:5.2f}%'.format(i,lbl,1000.val,100val/sumT))
3542
3543	def validateAtomDrawType(typ,generalData={}):
3544	'''Confirm that the selected Atom drawing type is valid for the current
3545	phase. If not, use 'vdW balls'. This is currently used only for setting a
3546	default when atoms are added to the atoms draw list.
3547	'''
3548	if typ in ('lines','vdW balls','sticks','balls & sticks','ellipsoids'):
3549	return typ
3550	# elif generalData.get('Type','') == 'macromolecular':
3551	# if typ in ('backbone',):
3552	# return typ
3553	return 'vdW balls'
3554
3555	if __name__ == "__main__":
3556	# test variable descriptions
3557	for var in '0::Afrac:*',':1:Scale','1::dAx:0','::undefined':
3558	v = var.split(':')[2]
3559	print(var+':\t', getDescr(v),getVarStep(v))
3560	import sys; sys.exit()
3561	# test equation evaluation
3562	def showEQ(calcobj):
3563	print (50*'=')
3564	print (calcobj.eObj.expression+'='+calcobj.EvalExpression())
3565	for v in sorted(calcobj.varLookup):
3566	print (" "+v+'='+calcobj.exprDict[v]+'='+calcobj.varLookup[v])
3567	# print ' Derivatives'
3568	# for v in calcobj.derivStep.keys():
3569	# print ' d(Expr)/d('+v+') =',calcobj.EvalDeriv(v)
3570
3571	obj = ExpressionObj()
3572
3573	obj.expression = "A*np.exp(B)"
3574	obj.assgnVars = {'B': '0::Afrac:1'}
3575	obj.freeVars = {'A': [u'A', 0.5, True]}
3576	#obj.CheckVars()
3577	calcobj = ExpressionCalcObj(obj)
3578
3579	obj1 = ExpressionObj()
3580	obj1.expression = "A*np.exp(B)"
3581	obj1.assgnVars = {'B': '0::Afrac:*'}
3582	obj1.freeVars = {'A': [u'Free Prm A', 0.5, True]}
3583	#obj.CheckVars()
3584	calcobj1 = ExpressionCalcObj(obj1)
3585
3586	obj2 = ExpressionObj()
3587	obj2.distance_stuff = np.array([[0,1],[1,-1]])
3588	obj2.expression = "Dist(1,2)"
3589	GSASIIpath.InvokeDebugOpts()
3590	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
3591	calcobj2 = ExpressionCalcObj(obj2)
3592	calcobj2.SetupCalc(parmDict2)
3593	showEQ(calcobj2)
3594
3595	parmDict1 = {'0::Afrac:0':1.0, '0::Afrac:1': 1.0}
3596	print ('\nDict = '+parmDict1)
3597	calcobj.SetupCalc(parmDict1)
3598	showEQ(calcobj)
3599	calcobj1.SetupCalc(parmDict1)
3600	showEQ(calcobj1)
3601
3602	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
3603	print ('Dict = '+parmDict2)
3604	calcobj.SetupCalc(parmDict2)
3605	showEQ(calcobj)
3606	calcobj1.SetupCalc(parmDict2)
3607	showEQ(calcobj1)
3608	calcobj2.SetupCalc(parmDict2)
3609	showEQ(calcobj2)

Note: See TracBrowser for help on using the repository browser.

Download in other formats: