Context Navigation

source: trunk/GSASIIobj.py @ 5164

Last change on this file since 5164 was 5148, checked in by toby, 3 years ago
add/fix comments
Property svn:eol-style set to `native` Property svn:keywords set to `Date Author Revision URL Id`
File size: 170.3 KB

Line
1	# -- coding: utf-8 --
2	#GSASIIobj - data objects for GSAS-II
3	########### SVN repository information ###################
4	# $Date: 2022-01-19 22:56:37 +0000 (Wed, 19 Jan 2022) $
5	# $Author: toby $
6	# $Revision: 5148 $
7	# $URL: trunk/GSASIIobj.py $
8	# $Id: GSASIIobj.py 5148 2022-01-19 22:56:37Z toby $
9	########### SVN repository information ###################
10
11	'''
12	GSASIIobj: Data objects
13	=========================
14
15	This module defines and/or documents the data structures used in GSAS-II, as well
16	as provides misc. support routines.
17
18	.. Next command allows \\AA to be used in HTML
19
20	.. only:: html
21
22	:math:`\\require{mediawiki-texvc}`
23
24	.. _Constraints_table:
25
26	.. index::
27	single: Constraints object description
28	single: Data object descriptions; Constraints
29
30	Constraints Tree Item
31	----------------------
32
33	Constraints are stored in a dict, separated into groups.
34	Note that parameter are named in the following pattern,
35	p:h:<var>:n, where p is the phase number, h is the histogram number
36	<var> is a variable name and n is the parameter number.
37	If a parameter does not depend on a histogram or phase or is unnumbered, that
38	number is omitted.
39	Note that the contents of each dict item is a List where each element in the
40	list is a :ref:`constraint definition objects <Constraint_definitions_table>`.
41	The constraints in this form are converted in
42	:func:`GSASIImapvars.ProcessConstraints` to the form used in :mod:`GSASIImapvars`
43
44	The keys in the Constraints dict are:
45
46	.. tabularcolumns:: \|l\|p{4.5in}\|
47
48	========== ====================================================
49	key explanation
50	========== ====================================================
51	Hist This specifies a list of constraints on
52	histogram-related parameters,
53	which will be of form :h:<var>:n.
54	HAP This specifies a list of constraints on parameters
55	that are defined for every histogram in each phase
56	and are of form p:h:<var>:n.
57	Phase This specifies a list of constraints on phase
58	parameters,
59	which will be of form p::<var>:n.
60	Global This specifies a list of constraints on parameters
61	that are not tied to a histogram or phase and
62	are of form ::<var>:n
63	========== ====================================================
64
65	.. _Constraint_definitions_table:
66
67	.. index::
68	single: Constraint definition object description
69	single: Data object descriptions; Constraint Definition
70
71	Each constraint is defined as an item in a list. Each constraint is of form::
72
73	[[<mult1>, <var1>], [<mult2>, <var2>],..., <fixedval>, <varyflag>, <constype>]
74
75	Where the variable pair list item containing two values [<mult>, <var>], where:
76
77	* <mult> is a multiplier for the constraint (float)
78	* <var> a :class:`G2VarObj` object. (Note that in very old .gpx files this might be a str with a variable name of form 'p:h:name[:at]')
79
80	Note that the last three items in the list play a special role:
81
82	* <fixedval> is the fixed value for a `constant equation` (``constype=c``)
83	constraint or is None. For a `New variable` (``constype=f``) constraint,
84	a variable name can be specified as a str (used for externally
85	generated constraints)
86	* <varyflag> is True or False for `New variable` (``constype=f``) constraints
87	or is None. This indicates if this variable should be refined.
88	* <constype> is one of four letters, 'e', 'c', 'h', 'f' that determines the type of constraint:
89
90	* 'e' defines a set of equivalent variables. Only the first variable is refined (if the
91	appropriate refine flag is set) and and all other equivalent variables in the list
92	are generated from that variable, using the appropriate multipliers.
93	* 'c' defines a constraint equation of form,
94	:math:`m_1 \\times var_1 + m_2 \\times var_2 + ... = c`
95	* 'h' defines a variable to hold (not vary). Any variable on this list is not varied,
96	even if its refinement flag is set. Only one [mult,var] pair is allowed in a hold
97	constraint and the mult value is ignored.
98	This is of particular value when needing to hold one or more variables where a
99	single flag controls a set of variables such as, coordinates,
100	the reciprocal metric tensor or anisotropic displacement parameter.
101	* 'f' defines a new variable (function) according to relationship
102	:math:`newvar = m_1 \\times var_1 + m_2 \\times var_2 + ...`
103
104	.. _Covariance_table:
105
106	.. index::
107	single: Covariance description
108	single: Data object descriptions; Covariance
109
110	Covariance Tree Item
111	--------------------
112
113	The Covariance tree item has results from the last least-squares run. They
114	are stored in a dict with these keys:
115
116	.. tabularcolumns:: \|l\|l\|p{4in}\|
117
118	============= =============== ====================================================
119	key sub-key explanation
120	============= =============== ====================================================
121	newCellDict \\ (dict) ith lattice parameters computed by
122	:func:`GSASIIstrMath.GetNewCellParms`
123	title \\ (str) Name of gpx file(?)
124	variables \\ (list) Values for all N refined variables
125	(list of float values, length N,
126	ordered to match varyList)
127	sig \\ (list) Uncertainty values for all N refined variables
128	(list of float values, length N,
129	ordered to match varyList)
130	varyList \\ (list of str values, length N) List of directly refined variables
131
132	newAtomDict \\ (dict) atom position values computed in
133	:func:`GSASIIstrMath.ApplyXYZshifts`
134	Rvals \\ (dict) R-factors, GOF, Marquardt value for last
135	refinement cycle
136	\\ Nobs (int) Number of observed data points
137	\\ Rwp (float) overall weighted profile R-factor (%)
138	\\ chisq (float) :math:`\\sum w*(I_{obs}-I_{calc})^2`
139	for all data.
140	Note: this is not the reduced :math:`\\chi^2`.
141	\\ lamMax (float) Marquardt value applied to Hessian diagonal
142	\\ GOF (float) The goodness-of-fit, aka square root of
143	the reduced chi squared.
144	covMatrix \\ (np.array) The (NxN) covVariance matrix
145	============= =============== ====================================================
146
147	.. _Phase_table:
148
149	.. index::
150	single: Phase object description
151	single: Data object descriptions; Phase
152
153	Phase Tree Items
154	----------------
155
156	Phase information is stored in the GSAS-II data tree as children of the
157	Phases item in a dict with keys:
158
159	.. tabularcolumns:: \|l\|l\|p{4in}\|
160
161	========== =============== =====================================================================================================
162	key sub-key explanation
163	========== =============== =====================================================================================================
164	General \\ (dict) Overall information for the phase
165	\\ 3Dproj (list of str) projections for 3D pole distribution plots
166	\\ AngleRadii (list of floats) Default radius for each atom used to compute
167	interatomic angles
168	\\ AtomMass (list of floats) Masses for atoms
169	\\ AtomPtrs (list of int) four locations (cx,ct,cs & cu) to use to pull info
170	from the atom records
171	\\ AtomTypes (llist of str) Atom types
172	\\ BondRadii (list of floats) Default radius for each atom used to compute
173	interatomic distances
174	\\ Cell Unit cell parameters & ref. flag
175	(list with 8 items. All but first item are float.)
176
177	0: cell refinement flag (True/False),
178	1-3: a, b, c, (:math:`\\AA`)
179	4-6: alpha, beta & gamma, (degrees)
180	7: volume (:math:`\\AA^3`)
181	\\ Color (list of (r,b,g) triplets) Colors for atoms
182	\\ Compare (dict) Polygon comparison parameters
183	\\ Data plot type (str) data plot type ('Mustrain', 'Size' or
184	'Preferred orientation') for powder data
185	\\ DisAglCtls (dDict) with distance/angle search controls,
186	which has keys 'Name', 'AtomTypes',
187	'BondRadii', 'AngleRadii' which are as above
188	except are possibly edited. Also contains
189	'Factors', which is a 2 element list with
190	a multiplier for bond and angle search range
191	[typically (0.85,0.85)].
192	\\ F000X (float) x-ray F(000) intensity
193	\\ F000N (float) neutron F(000) intensity
194	\\ Flip (dict) Charge flip controls
195	\\ HydIds (dict) geometrically generated hydrogen atoms
196	\\ Isotope (dict) Isotopes for each atom type
197	\\ Isotopes (dict) Scattering lengths for each isotope
198	combination for each element in phase
199	\\ MCSA controls (dict) Monte Carlo-Simulated Annealing controls
200	\\ Map (dict) Map parameters
201	\\ Mass (float) Mass of unit cell contents in g/mol
202	\\ Modulated (bool) True if phase modulated
203	\\ Mydir (str) Directory of current .gpx file
204	\\ Name (str) Phase name
205	\\ NoAtoms (dict) Number of atoms per unit cell of each type
206	\\ POhkl (list) March-Dollase preferred orientation direction
207	\\ Pawley dmin (float) maximum Q (as d-space) to use for Pawley extraction
208	\\ Pawley dmax (float) minimum Q (as d-space) to use for Pawley extraction
209	\\ Pawley neg wt (float) Restraint value for negative Pawley intensities
210	\\ SGData (object) Space group details as a
211	:ref:`space group (SGData) <SGData_table>`
212	object, as defined in :func:`GSASIIspc.SpcGroup`.
213	\\ SH Texture (dict) Spherical harmonic preferred orientation parameters
214	\\ Super (int) dimension of super group (0,1 only)
215	\\ Type (str) phase type (e.g. 'nuclear')
216	\\ Z (dict) Atomic numbers for each atom type
217	\\ doDysnomia (bool) flag for max ent map modification via Dysnomia
218	\\ doPawley (bool) Flag for Pawley intensity extraction
219	\\ vdWRadii (dict) Van der Waals radii for each atom type
220	ranId \\ (int) unique random number Id for phase
221	pId \\ (int) Phase Id number for current project.
222	Atoms \\ (list of lists) Atoms in phase as a list of lists. The outer list
223	is for each atom, the inner list contains varying
224	items depending on the type of phase, see
225	the :ref:`Atom Records <Atoms_table>` description.
226	Drawing \\ (dict) Display parameters
227	\\ Atoms (list of lists) with an entry for each atom that is drawn
228	\\ Plane (list) Controls for contour density plane display
229	\\ Quaternion (4 element np.array) Viewing quaternion
230	\\ Zclip (float) clipping distance in :math:`\\AA`
231	\\ Zstep (float) Step to de/increase Z-clip
232	\\ atomPtrs (list) positions of x, type, site sym, ADP flag in Draw Atoms
233	\\ backColor (list) background for plot as and R,G,B triplet
234	(default = [0, 0, 0], black).
235	\\ ballScale (float) Radius of spheres in ball-and-stick display
236	\\ bondList (dict) Bonds
237	\\ bondRadius (float) Radius of binds in :math:`\\AA`
238	\\ cameraPos (float) Viewing position in :math:`\\AA` for plot
239	\\ contourLevel (float) map contour level in :math:`e/\\AA^3`
240	\\ contourMax (float) map contour maximum
241	\\ depthFog (bool) True if use depthFog on plot - set currently as False
242	\\ ellipseProb (float) Probability limit for display of thermal
243	ellipsoids in % .
244	\\ magMult (float) multiplier for magnetic moment arrows
245	\\ mapSize (float) x & y dimensions of contourmap (fixed internally)
246	\\ modelView (4,4 array) from openGL drawing transofmation matrix
247	\\ oldxy (list with two floats) previous view point
248	\\ radiusFactor (float) Distance ratio for searching for bonds. Bonds
249	are located that are within r(Ra+Rb) and (Ra+Rb)/r
250	where Ra and Rb are the atomic radii.
251	\\ selectedAtoms (list of int values) List of selected atoms
252	\\ showABC (bool) Flag to show view point triplet. True=show.
253	\\ showHydrogen (bool) Flag to control plotting of H atoms.
254	\\ showRigidBodies (bool) Flag to highlight rigid body placement
255	\\ showSlice (bool) flag to show contour map
256	\\ sizeH (float) Size ratio for H atoms
257	\\ unitCellBox (bool) Flag to control display of the unit cell.
258	\\ vdwScale (float) Multiplier of van der Waals radius for display of vdW spheres.
259	\\ viewDir (np.array with three floats) cartesian viewing direction
260	\\ viewPoint (list of lists) First item in list is [x,y,z]
261	in fractional coordinates for the center of
262	the plot. Second item list of previous & current
263	atom number viewed (may be [0,0])
264	ISODISTORT \\ (dict) contains controls for running ISODISTORT and results from it
265	\\ ISOmethod (int) ISODISTORT method (currently 1 or 4; 2 & 3 not implemented in GSAS-II)
266	\\ ParentCIF (str) parent cif file name for ISODISTORT method 4
267	\\ ChildCIF (str) child cif file name for ISODISTORT method 4
268	\\ SGselect (dict) selection list for lattice types in radio result from ISODISTORT method 1
269	\\ selection (int) chosen selection from radio
270	\\ radio (list) results from ISODISTORT method 1
271	\\ ChildMatrix (3x3 array) transformation matrix for method 3 (not currently used)
272	\\ ChildSprGp (str) child space group for method 3 (not currently used)
273	\\ ChildCell (str) cell ordering for nonstandard orthorhombic ChildSprGrp in method 3 (not currently used)
274	\\ G2ModeList (list) ISODISTORT mode names
275	\\ modeDispl (list) distortion mode values; refinable parameters
276	\\ ISOmodeDispl (list) distortion mode values as determined in method 4 by ISODISTORT
277	\\ NormList (list) ISODISTORT normalization values; to convert mode value to fractional coordinate dsplacement
278	\\ G2parentCoords (list) full set of parent structure coordinates transformed to child structure; starting basis for mode displacements
279	\\ G2VarList (list)
280	\\ IsoVarList (list)
281	\\ G2coordOffset (list) only adjustible set of parent structure coordinates
282	\\ G2OccVarList (list)
283	\\ Var2ModeMatrix (array) atom variable to distortion mode transformation
284	\\ Mode2VarMatrix (array) distortion mode to atom variable transformation
285	\\ rundata (dict) saved input information for use by ISODISTORT method 1
286
287	RBModels \\ Rigid body assignments (note Rigid body definitions
288	are stored in their own main top-level tree entry.)
289	RMC \\ (dict) RMCProfile, PDFfit & fullrmc controls
290	Pawley ref \\ (list) Pawley reflections
291	Histograms \\ (dict of dicts) The key for the outer dict is
292	the histograms tied to this phase. The inner
293	dict contains the combined phase/histogram
294	parameters for items such as scale factors,
295	size and strain parameters. The following are the
296	keys to the inner dict. (dict)
297	\\ Babinet (dict) For protein crystallography. Dictionary with two
298	entries, 'BabA', 'BabU'
299	\\ Extinction (list of float, bool) Extinction parameter
300	\\ Flack (list of [float, bool]) Flack parameter & refine flag
301	\\ HStrain (list of two lists) Hydrostatic strain. The first is
302	a list of the HStrain parameters (1, 2, 3, 4, or 6
303	depending on unit cell), the second is a list of boolean
304	refinement parameters (same length)
305	\\ Histogram (str) The name of the associated histogram
306	\\ Layer Disp (list of [float, bool]) Layer displacement in beam direction & refine flag
307	\\ LeBail (bool) Flag for LeBail extraction
308	\\ Mustrain (list) Microstrain parameters, in order:
309
310	0. Type, one of u'isotropic', u'uniaxial', u'generalized'
311	1. Isotropic/uniaxial parameters - list of 3 floats
312	2. Refinement flags - list of 3 bools
313	3. Microstrain axis - list of 3 ints, [h, k, l]
314	4. Generalized mustrain parameters - list of 2-6 floats, depending on space group
315	5. Generalized refinement flags - list of bools, corresponding to the parameters of (4)
316	\\ Pref.Ori. (list) Preferred Orientation. List of eight parameters.
317	Items marked SH are only used for Spherical Harmonics.
318
319	0. (str) Type, 'MD' for March-Dollase or 'SH' for Spherical Harmonics
320	1. (float) Value
321	2. (bool) Refinement flag
322	3. (list) Preferred direction, list of ints, [h, k, l]
323	4. (int) SH - number of terms
324	5. (dict) SH -
325	6. (list) SH
326	7. (float) SH
327	\\ Scale (list of [float, bool]) Phase fraction & refine flag
328	\\ Size List of crystallite size parameters, in order:
329
330	0. (str) Type, one of u'isotropic', u'uniaxial', u'ellipsoidal'
331	1. (list) Isotropic/uniaxial parameters - list of 3 floats
332	2. (list) Refinement flags - list of 3 bools
333	3. (list) Size axis - list of 3 ints, [h, k, l]
334	4. (list) Ellipsoidal size parameters - list of 6 floats
335	5. (list) Ellipsoidal refinement flags - list of bools, corresponding to the parameters of (4)
336	\\ Use (bool) True if this histogram is to be used in refinement
337	MCSA \\ (dict) Monte-Carlo simulated annealing parameters
338	========== =============== =====================================================================================================
339
340	.. _RBData_table:
341
342	.. index::
343	single: Rigid Body Data description
344	single: Data object descriptions; Rigid Body Data
345
346	Rigid Body Objects
347	------------------
348
349	Rigid body descriptions are available for two types of rigid bodies: 'Vector'
350	and 'Residue'. Vector rigid bodies are developed by a sequence of translations each
351	with a refinable magnitude and Residue rigid bodies are described as Cartesian coordinates
352	with defined refinable torsion angles.
353
354	.. tabularcolumns:: \|l\|l\|p{4in}\|
355
356	========== =============== ====================================================
357	key sub-key explanation
358	========== =============== ====================================================
359	Vector RBId (dict of dict) vector rigid bodies
360	\\ AtInfo (dict) Drad, Color: atom drawing radius & color for each atom type
361	\\ RBname (str) Name assigned by user to rigid body
362	\\ VectMag (list) vector magnitudes in :math:`\\AA`
363	\\ rbXYZ (list of 3 float Cartesian coordinates for Vector rigid body )
364	\\ rbRef (list of 3 int & 1 bool) 3 assigned reference atom nos. in rigid body for origin
365	definition, use center of atoms flag
366	\\ VectRef (list of bool refinement flags for VectMag values )
367	\\ rbTypes (list of str) Atom types for each atom in rigid body
368	\\ rbVect (list of lists) Cartesian vectors for each translation used to build rigid body
369	\\ useCount (int) Number of times rigid body is used in any structure
370	Residue RBId (dict of dict) residue rigid bodies
371	\\ AtInfo (dict) Drad, Color: atom drawing radius & color for each atom type
372	\\ RBname (str) Name assigned by user to rigid body
373	\\ rbXYZ (list of 3 float) Cartesian coordinates for Residue rigid body
374	\\ rbTypes (list of str) Atom types for each atom in rigid body
375	\\ atNames (list of str) Names of each atom in rigid body (e.g. C1,N2...)
376	\\ rbRef (list of 3 int & 1 bool) 3 assigned reference atom nos. in rigid body for origin
377	definition, use center of atoms flag
378	\\ rbSeq (list) Orig,Piv,angle,Riding : definition of internal rigid body
379	torsion; origin atom (int), pivot atom (int), torsion angle (float),
380	riding atoms (list of int)
381	\\ SelSeq (int,int) used by SeqSizer to identify objects
382	\\ useCount (int)Number of times rigid body is used in any structure
383	RBIds \\ (dict) unique Ids generated upon creation of each rigid body
384	\\ Vector (list) Ids for each Vector rigid body
385	\\ Residue (list) Ids for each Residue rigid body
386	========== =============== ====================================================
387
388	.. _SGData_table:
389
390	.. index::
391	single: Space Group Data description
392	single: Data object descriptions; Space Group Data
393
394	Space Group Objects
395	-------------------
396
397	Space groups are interpreted by :func:`GSASIIspc.SpcGroup`
398	and the information is placed in a SGdata object
399	which is a dict with these keys. Magnetic ones are marked "mag"
400
401	.. tabularcolumns:: \|l\|p{4.5in}\|
402
403	========== ========================================================================================
404	key explanation
405	========== ========================================================================================
406	BNSlattsym mag - (str) BNS magnetic space group symbol and centering vector
407	GenFlg mag - (list) symmetry generators indices
408	GenSym mag - (list) names for each generator
409	MagMom mag - (list) "time reversals" for each magnetic operator
410	MagPtGp mag - (str) Magnetic point group symbol
411	MagSpGrp mag - (str) Magnetic space group symbol
412	OprNames mag - (list) names for each space group operation
413	SGCen (np.array) Symmetry cell centering vectors. A (n,3) np.array
414	of centers. Will always have at least one row: ``np.array([[0, 0, 0]])``
415	SGFixed (bool) Only True if phase mported from a magnetic cif file
416	then the space group can not be changed by the user because
417	operator set from cif may be nonstandard
418	SGGen (list) generators
419	SGGray (bool) True if space group is a gray group (incommensurate magnetic structures)
420	SGInv (bool) True if centrosymmetric, False if not
421	SGLatt (str)Lattice centering type. Will be one of
422	P, A, B, C, I, F, R
423	SGLaue (str) one of the following 14 Laue classes:
424	-1, 2/m, mmm, 4/m, 4/mmm, 3R,
425	3mR, 3, 3m1, 31m, 6/m, 6/mmm, m3, m3m
426	SGOps (list) symmetry operations as a list of form
427	``[[M1,T1], [M2,T2],...]``
428	where :math:`M_n` is a 3x3 np.array
429	and :math:`T_n` is a length 3 np.array.
430	Atom coordinates are transformed where the
431	Asymmetric unit coordinates [X is (x,y,z)]
432	are transformed using
433	:math:`X^\\prime = M_n*X+T_n`
434	SGPolax (str) Axes for space group polarity. Will be one of
435	'', 'x', 'y', 'x y', 'z', 'x z', 'y z',
436	'xyz'. In the case where axes are arbitrary
437	'111' is used (P 1, and ?).
438	SGPtGrp (str) Point group of the space group
439	SGUniq unique axis if monoclinic. Will be
440	a, b, or c for monoclinic space groups.
441	Will be blank for non-monoclinic.
442	SGSpin mag - (list) of spin flip operatiors (+1 or -1) for the space group operations
443	SGSys (str) symmetry unit cell: type one of
444	'triclinic', 'monoclinic', 'orthorhombic',
445	'tetragonal', 'rhombohedral', 'trigonal',
446	'hexagonal', 'cubic'
447	SSGK1 (list) Superspace multipliers
448	SpGrp (str) space group symbol
449	SpnFlp mag - (list) Magnetic spin flips for every magnetic space group operator
450	========== ========================================================================================
451
452	.. _SSGData_table:
453
454	.. index::
455	single: Superspace Group Data description
456	single: Data object descriptions; Superspace Group Data
457
458	Superspace groups [3+1] are interpreted by :func:`GSASIIspc.SSpcGroup`
459	and the information is placed in a SSGdata object
460	which is a dict with these keys:
461
462	.. tabularcolumns:: \|l\|p{4.5in}\|
463
464	========== ====================================================
465	key explanation
466	========== ====================================================
467	SSGCen (list) 4D cell centering vectors [0,0,0,0] at least
468	SSGK1 (list) Superspace multipliers
469	SSGOps (list) 4D symmetry operations as [M,T] so that M*x+T = x'
470	SSpGrp (str) superspace group symbol extension to space group
471	symbol, accidental spaces removed
472	modQ (list) modulation/propagation vector
473	modSymb (list of str) Modulation symbols
474	========== ====================================================
475
476
477	Phase Information
478	--------------------
479
480	.. index::
481	single: Phase information record description
482
483	Phase information is placed in one of the following keys:
484
485	.. tabularcolumns:: \|l\|p{4.5in}\|
486
487	========== ==============================================================
488	key explanation
489	========== ==============================================================
490	General Overall information about a phase
491	Histograms Information about each histogram linked to the
492	current phase as well as parameters that
493	are defined for each histogram and phase
494	(such as sample peak widths and preferred
495	orientation parameters.
496	Atoms Contains a list of atoms, as described in the
497	:ref:`Atom Records <Atoms_table>` description.
498	Drawing Parameters that determine how the phase is
499	displayed, including a list of atoms to be
500	included, as described in the
501	:ref:`Drawing Atom Records <Drawing_atoms_table>`
502	description
503	MCSA Monte-Carlo simulated annealing parameters
504	pId The index of each phase in the project, numbered
505	starting at 0
506	ranId An int value with a unique value for each phase
507	RBModels A list of dicts with parameters for each
508	rigid body inserted into the current phase,
509	as defined in the
510	:ref:`Rigid Body Insertions <Rigid_Body_Insertions>`.
511	Note that the rigid bodies are defined as
512	:ref:`Rigid Body Objects <RBData_table>`
513	RMC PDF modeling parameters
514	Pawley ref Pawley refinement parameters
515
516	========== ==============================================================
517
518	.. _Atoms_table:
519
520	.. index::
521	single: Atoms record description
522	single: Data object descriptions; Atoms record
523
524	--------------------
525	Atom Records
526	--------------------
527
528	If ``phasedict`` points to the phase information in the data tree, then
529	atoms are contained in a list of atom records (list) in
530	``phasedict['Atoms']``. Also needed to read atom information
531	are four pointers, ``cx,ct,cs,cia = phasedict['General']['AtomPtrs']``,
532	which define locations in the atom record, as shown below. Items shown are
533	always present; additional ones for macromolecular phases are marked 'mm',
534	and those for magnetic structures are marked 'mg'
535
536	.. tabularcolumns:: \|l\|p{4.5in}\|
537
538	============== ====================================================
539	location explanation
540	============== ====================================================
541	ct-4 mm - (str) residue number
542	ct-3 mm - (str) residue name (e.g. ALA)
543	ct-2 mm - (str) chain label
544	ct-1 (str) atom label
545	ct (str) atom type
546	ct+1 (str) refinement flags; combination of 'F', 'X', 'U', 'M'
547	cx,cx+1,cx+2 (3 floats) the x,y and z coordinates
548	cx+3 (float) site occupancy
549	cx+4,cx+5,cx+6 mg - (list) atom magnetic moment along a,b,c in Bohr magnetons
550	cs (str) site symmetry
551	cs+1 (int) site multiplicity
552	cia (str) ADP flag: Isotropic ('I') or Anisotropic ('A')
553	cia+1 (float) Uiso
554	cia+2...cia+7 (6 floats) U11, U22, U33, U12, U13, U23
555	atom[cia+8] (int) unique atom identifier
556
557	============== ====================================================
558
559	.. _Drawing_atoms_table:
560
561	.. index::
562	single: Drawing atoms record description
563	single: Data object descriptions; Drawing atoms record
564
565	----------------------------
566	Drawing Atom Records
567	----------------------------
568
569	If ``phasedict`` points to the phase information in the data tree, then
570	drawing atoms are contained in a list of drawing atom records (list) in
571	``phasedict['Drawing']['Atoms']``. Also needed to read atom information
572	are four pointers, ``cx,ct,cs,ci = phasedict['Drawing']['AtomPtrs']``,
573	which define locations in the atom record, as shown below. Items shown are
574	always present; additional ones for macromolecular phases are marked 'mm',
575	and those for magnetic structures are marked 'mg'
576
577	.. tabularcolumns:: \|l\|p{4.5in}\|
578
579	============== ===================================================================================
580	location explanation
581	============== ===================================================================================
582	ct-4 mm - (str) residue number
583	ct-3 mm - (str) residue name (e.g. ALA)
584	ct-2 mm - (str) chain label
585	ct-1 (str) atom label
586	ct (str) atom type
587	cx,cx+1,cx+2 (3 floats) the x,y and z coordinates
588	cx+3,cx+4,cx+5 mg - (3 floats) atom magnetic moment along a,b,c in Bohr magnetons
589	cs-1 (str) Sym Op symbol; sym. op number + unit cell id (e.g. '1,0,-1')
590	cs (str) atom drawing style; e.g. 'balls & sticks'
591	cs+1 (str) atom label style (e.g. 'name')
592	cs+2 (int) atom color (RBG triplet)
593	cs+3 (str) ADP flag: Isotropic ('I') or Anisotropic ('A')
594	cs+4 (float) Uiso
595	cs+5...cs+11 (6 floats) U11, U22, U33, U12, U13, U23
596	ci (int) unique atom identifier; matches source atom Id in Atom Records
597	============== ===================================================================================
598
599	.. _Rigid_Body_Insertions:
600
601	----------------------------
602	Rigid Body Insertions
603	----------------------------
604
605	If ``phasedict`` points to the phase information in the data tree, then
606	rigid body information is contained in list(s) in
607	``phasedict['RBModels']['Residue']`` and/or ``phasedict['RBModels']['Vector']``
608	for each rigid body inserted into the current phase.
609
610	.. tabularcolumns:: \|l\|p{4.5in}\|
611
612	============== ===================================================================================
613	key explanation
614	============== ===================================================================================
615	fixOrig Should the origin be fixed (when editing, not the refinement flag)
616	Ids Ids for assignment of atoms in the rigid body
617	numChain Chain number for macromolecular fits
618	Orient Orientation of the RB as a quaternion and a refinement flag (' ', 'A' or 'AV')
619	OrientVec Orientation of the RB expressed as a vector and azimuthal rotation angle
620	Orig Origin of the RB in fractional coordinates and refinement flag (bool)
621	RBId References the unique ID of a rigid body in the
622	:ref:`Rigid Body Objects <RBData_table>`
623	RBname The name for the rigid body (str)
624	AtomFrac The atom fractions for the rigid body
625	ThermalMotion The thermal motion description for the rigid body, which includes a choice for
626	the model and can include TLS parameters or an overall Uiso value.
627	Torsions Defines the torsion angle and refinement flag for each torsion defined in
628	the :ref:`Rigid Body Object <RBData_table>`
629	============== ===================================================================================
630
631	.. _Powder_table:
632
633	.. index::
634	single: Powder data object description
635	single: Data object descriptions; Powder Data
636
637	Powder Diffraction Tree Items
638	-----------------------------
639
640	Every powder diffraction histogram is stored in the GSAS-II data tree
641	with a top-level entry named beginning with the string "PWDR ". The
642	diffraction data for that information are directly associated with
643	that tree item and there are a series of children to that item. The
644	routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
645	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
646	load this information into a dictionary where the child tree name is
647	used as a key, and the information in the main entry is assigned
648	a key of ``Data``, as outlined below.
649
650	.. tabularcolumns:: \|p{1in}\|p{1in}\|p{4in}\|
651
652	====================== =============== ===========================================================
653	key sub-key explanation
654	====================== =============== ===========================================================
655	Comments \\ (list of str) Text strings extracted from the original powder
656	data header. These cannot be changed by the user;
657	it may be empty.
658	Limits \\ (list) two two element lists, as [[Ld,Hd],[L,H]]
659	where L and Ld are the current and default lowest
660	two-theta value to be used and
661	where H and Hd are the current and default highest
662	two-theta value to be used.
663	Reflection Lists \\ (dict of dicts) with an entry for each phase in the
664	histogram. The contents of each dict item
665	is a dict containing reflections, as described in
666	the :ref:`Powder Reflections <PowderRefl_table>`
667	description.
668	Instrument Parameters \\ (dict) The instrument parameters uses different dicts
669	for the constant wavelength (CW) and time-of-flight (TOF)
670	cases. See below for the descriptions of each.
671	wtFactor \\ (float) A weighting factor to increase or decrease
672	the leverage of data in the histogram .
673	A value of 1.0 weights the data with their
674	standard uncertainties and a larger value
675	increases the weighting of the data (equivalent
676	to decreasing the uncertainties).
677	Sample Parameters \\ (dict) Parameters that describe how
678	the data were collected, as listed
679	below. Refinable parameters are a list containing
680	a float and a bool, where the second value
681	specifies if the value is refined, otherwise
682	the value is a float unless otherwise noted.
683	\\ Scale The histogram scale factor (refinable)
684	\\ Absorption The sample absorption coefficient as
685	:math:`\\mu r` where r is the radius
686	(refinable). Only valid for Debye-Scherrer geometry.
687	\\ SurfaceRoughA Surface roughness parameter A as defined by
688	Surotti, J. Appl. Cryst, 5, 325-331, 1972.
689	(refinable - only valid for Bragg-Brentano geometry)
690	\\ SurfaceRoughB Surface roughness parameter B (refinable -
691	only valid for Bragg-Brentano geometry)
692	\\ DisplaceX, Sample displacement from goniometer center
693	DisplaceY where Y is along the beam direction and
694	X is perpendicular. Units are :math:`\\mu m`
695	(refinable).
696	\\ Phi, Chi, Goniometer sample setting angles, in degrees.
697	Omega
698	\\ Gonio. radius Radius of the diffractometer in mm
699	\\ InstrName (str) A name for the instrument, used in preparing
700	a CIF .
701	\\ Force, Variables that describe how the measurement
702	Temperature, was performed. Not used directly in
703	Humidity, any computations.
704	Pressure,
705	Voltage
706	\\ ranId (int) The random-number Id for the histogram
707	(same value as where top-level key is ranId)
708	\\ Type (str) Type of diffraction data, may be 'Debye-Scherrer'
709	or 'Bragg-Brentano' .
710	hId \\ (int) The number assigned to the histogram when
711	the project is loaded or edited (can change)
712	ranId \\ (int) A random number id for the histogram
713	that does not change
714	Background \\ (list) The background is stored as a list with where
715	the first item in the list is list and the second
716	item is a dict. The list contains the background
717	function and its coefficients; the dict contains
718	Debye diffuse terms and background peaks.
719	(TODO: this needs to be expanded.)
720	Data \\ (list) The data consist of a list of 6 np.arrays
721	containing in order:
722
723	0. the x-postions (two-theta in degrees),
724	1. the intensity values (Yobs),
725	2. the weights for each Yobs value
726	3. the computed intensity values (Ycalc)
727	4. the background values
728	5. Yobs-Ycalc
729	====================== =============== ===========================================================
730
731	.. _CWPowder_table:
732
733	.. index::
734	single: Powder data CW Instrument Parameters
735
736	-----------------------------
737	CW Instrument Parameters
738	-----------------------------
739
740	Instrument Parameters are placed in a list of two dicts,
741	where the keys in the first dict are listed below. Note that the dict contents are different for
742	constant wavelength (CW) vs. time-of-flight (TOF) histograms.
743	The value for each item is a list containing three values: the initial value, the current value
744	and a refinement flag which can have a value of True, False or 0 where 0 indicates a value that
745	cannot be refined. The first and second values are floats unless otherwise noted.
746	Items not refined are noted as [*]
747
748	.. tabularcolumns:: \|l\|p{1in}\|p{4in}\|
749
750	======================== =============== ===========================================================
751	key sub-key explanation
752	======================== =============== ===========================================================
753	Instrument Parameters[0] Type [*] (str) Histogram type:
754	* 'PXC' for constant wavelength x-ray
755	* 'PNC' for constant wavelength neutron
756	\\ Bank [*] (int) Data set number in a multidata file (usually 1)
757	\\ Lam (float) Specifies a wavelength in :math:`\\AA`
758	\\ Lam1 [*] (float) Specifies the primary wavelength in
759	:math:`\\AA`, used in place of Lam
760	when an :math:`\\alpha_1, \\alpha_2`
761	source is used.
762	\\ Lam2 [*] (float) Specifies the secondary wavelength in
763	:math:`\\AA`, used with Lam1
764	\\ I(L2)/I(L1) (float) Ratio of Lam2 to Lam1, used with Lam1
765	\\ Zero (float) Two-theta zero correction in degrees
766	\\ Azimuth [*] (float) Azimuthal setting angle for data recorded with differing setting angles
767	\\ U, V, W (float) Cagliotti profile coefficients
768	for Gaussian instrumental broadening, where the
769	FWHM goes as
770	:math:`U \\tan^2\\theta + V \\tan\\theta + W`
771	\\ X, Y, Z (float) Cauchy (Lorentzian) instrumental broadening coefficients
772	\\ SH/L (float) Variant of the Finger-Cox-Jephcoat asymmetric
773	peak broadening ratio. Note that this is the
774	sum of S/L and H/L where S is
775	sample height, H is the slit height and
776	L is the goniometer diameter.
777	\\ Polariz. (float) Polarization coefficient.
778	Instrument Parameters[1] (empty dict)
779	======================== =============== ===========================================================
780
781	.. _TOFPowder_table:
782
783	.. index::
784	single: Powder data TOF Instrument Parameters
785
786	-----------------------------
787	TOF Instrument Parameters
788	-----------------------------
789
790	Instrument Parameters are also placed in a list of two dicts,
791	where the keys in each dict listed below, but here for
792	time-of-flight (TOF) histograms.
793	The value for each item is a list containing three values: the initial value, the current value
794	and a refinement flag which can have a value of True, False or 0 where 0 indicates a value that
795	cannot be refined. The first and second values are floats unless otherwise noted.
796	Items not refined are noted as [*]
797
798	.. tabularcolumns:: \|l\|p{1.5in}\|p{4in}\|
799
800	======================== =============== ===========================================================
801	key sub-key explanation
802	======================== =============== ===========================================================
803	Instrument Parameters[0] Type [*] (str) Histogram type:
804	* 'PNT' for time of flight neutron
805	\\ Bank (int) Data set number in a multidata file
806	\\ 2-theta [*] (float) Nominal scattering angle for the detector
807	\\ fltPath [*] (float) Total flight path source-sample-detector
808	\\ Azimuth [*] (float) Azimuth angle for detector right hand rotation
809	from horizontal away from source
810	\\ difC,difA, (float) Diffractometer constants for conversion of d-spacing to TOF
811	difB in microseconds
812	\\ Zero (float) Zero point offset (microseconds)
813	\\ alpha (float) Exponential rise profile coefficients
814	\\ beta-0 (float) Exponential decay profile coefficients
815	beta-1
816	beta-q
817	\\ sig-0 (float) Gaussian profile coefficients
818	sig-1
819	sig-2
820	sig-q
821	\\ X,Y,Z (float) Lorentzian profile coefficients
822	Instrument Parameters[1] Pdabc (list of 4 float lists) Originally created for use in gsas as optional tables
823	of d, alp, bet, d-true; for a reflection alpha & beta are obtained via interpolation
824	from the d-spacing and these tables. The d-true column is apparently unused.
825	======================== =============== ===========================================================
826
827
828	.. _PowderRefl_table:
829
830	.. index::
831	single: Powder reflection object description
832	single: Data object descriptions; Powder Reflections
833
834	Powder Reflection Data Structure
835	--------------------------------
836
837	For every phase in a histogram, the ``Reflection Lists`` value is a dict
838	one element of which is `'RefList'`, which is a np.array containing
839	reflections. The columns in that array are documented below.
840
841	========== ====================================================
842	index explanation
843	========== ====================================================
844	0,1,2 h,k,l (float)
845	3 (int) multiplicity
846	4 (float) d-space, :math:`\\AA`
847	5 (float) pos, two-theta
848	6 (float) sig, Gaussian width
849	7 (float) gam, Lorenzian width
850	8 (float) :math:`F_{obs}^2`
851	9 (float) :math:`F_{calc}^2`
852	10 (float) reflection phase, in degrees
853	11 (float) intensity correction for reflection, this times
854	:math:`F_{obs}^2` or :math:`F_{calc}^2` gives Iobs or Icalc
855	12 (float) Preferred orientation correction
856	13 (float) Transmission (absorption correction)
857	14 (float) Extinction correction
858	========== ====================================================
859
860	.. _Xtal_table:
861
862	.. index::
863	single: Single Crystal data object description
864	single: Data object descriptions; Single crystal data
865
866	Single Crystal Tree Items
867	-------------------------
868
869	Every single crystal diffraction histogram is stored in the GSAS-II data tree
870	with a top-level entry named beginning with the string "HKLF ". The
871	diffraction data for that information are directly associated with
872	that tree item and there are a series of children to that item. The
873	routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
874	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
875	load this information into a dictionary where the child tree name is
876	used as a key, and the information in the main entry is assigned
877	a key of ``Data``, as outlined below.
878
879	.. tabularcolumns:: \|l\|l\|p{4in}\|
880
881	====================== =============== ====================================================
882	key sub-key explanation
883	====================== =============== ====================================================
884	Data \\ (dict) that contains the
885	reflection table,
886	as described in the
887	:ref:`Single Crystal Reflections
888	<XtalRefl_table>`
889	description.
890
891	Instrument Parameters \\ (list) containing two dicts where the possible
892	keys in each dict are listed below. The value
893	for most items is a list containing two values:
894	the initial value, the current value.
895	The first and second
896	values are floats unless otherwise noted.
897	\\ Lam (two floats) Specifies a wavelength in :math:`\\AA`
898	\\ Type (two str values) Histogram type :
899	* 'SXC' for constant wavelength x-ray
900	* 'SNC' for constant wavelength neutron
901	* 'SNT' for time of flight neutron
902	\\ InstrName (str) A name for the instrument, used in preparing a CIF
903	wtFactor \\ (float) A weighting factor to increase or decrease
904	the leverage of data in the histogram.
905	A value of 1.0 weights the data with their
906	standard uncertainties and a larger value
907	increases the weighting of the data (equivalent
908	to decreasing the uncertainties).
909
910	hId \\ (int) The number assigned to the histogram when
911	the project is loaded or edited (can change)
912	ranId \\ (int) A random number id for the histogram
913	that does not change
914	====================== =============== ====================================================
915
916	.. _XtalRefl_table:
917
918	.. index::
919	single: Single Crystal reflection object description
920	single: Data object descriptions; Single Crystal Reflections
921
922	Single Crystal Reflection Data Structure
923	----------------------------------------
924
925	For every single crystal a histogram, the ``'Data'`` item contains
926	the structure factors as an np.array in item `'RefList'`.
927	The columns in that array are documented below.
928
929	.. tabularcolumns:: \|l\|p{4in}\|
930
931	========== ====================================================
932	index explanation
933	========== ====================================================
934	0,1,2 (float) h,k,l
935	3 (int) multiplicity
936	4 (float) d-space, :math:`\\AA`
937	5 (float) :math:`F_{obs}^2`
938	6 (float) :math:`\\sigma(F_{obs}^2)`
939	7 (float) :math:`F_{calc}^2`
940	8 (float) :math:`F_{obs}^2T`
941	9 (float) :math:`F_{calc}^2T`
942	10 (float) reflection phase, in degrees
943	11 (float) intensity correction for reflection, this times
944	:math:`F_{obs}^2` or :math:`F_{calc}^2`
945	gives Iobs or Icalc
946	========== ====================================================
947
948	.. _Image_table:
949
950	.. index::
951	image: Image data object description
952	image: Image object descriptions
953
954	Image Data Structure
955	--------------------
956
957	Every 2-dimensional image is stored in the GSAS-II data tree
958	with a top-level entry named beginning with the string "IMG ". The
959	image data are directly associated with that tree item and there
960	are a series of children to that item. The routines :func:`GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree`
961	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
962	load this information into a dictionary where the child tree name is
963	used as a key, and the information in the main entry is assigned
964	a key of ``Data``, as outlined below.
965
966	.. tabularcolumns:: \|l\|l\|p{4in}\|
967
968	====================== ====================== ====================================================
969	key sub-key explanation
970	====================== ====================== ====================================================
971	Comments \\ (list of str) Text strings extracted from the original image data
972	header or a metafile. These cannot be changed by
973	the user; it may be empty.
974	Image Controls azmthOff (float) The offset to be applied to an azimuthal
975	value. Accomodates
976	detector orientations other than with the detector
977	X-axis
978	horizontal.
979	\\ background image (list:str,float) The name of a tree item ("IMG ...") that is to be subtracted
980	during image integration multiplied by value. It must have the same size/shape as
981	the integrated image. NB: value < 0 for subtraction.
982	\\ calibrant (str) The material used for determining the position/orientation
983	of the image. The data is obtained from :func:`ImageCalibrants`
984	and UserCalibrants.py (supplied by user).
985	\\ calibdmin (float) The minimum d-spacing used during the last calibration run.
986	\\ calibskip (int) The number of expected diffraction lines skipped during the last
987	calibration run.
988	\\ center (list:floats) The [X,Y] point in detector coordinates (mm) where the direct beam
989	strikes the detector plane as determined by calibration. This point
990	does not have to be within the limits of the detector boundaries.
991	\\ centerAzm (bool) If True then the azimuth reported for the integrated slice
992	of the image is at the center line otherwise it is at the leading edge.
993	\\ color (str) The name of the colormap used to display the image. Default = 'Paired'.
994	\\ cutoff (float) The minimum value of I/Ib for a point selected in a diffraction ring for
995	calibration calculations. See pixLimit for details as how point is found.
996	\\ DetDepth (float) Coefficient for penetration correction to distance; accounts for diffraction
997	ring offset at higher angles. Optionally determined by calibration.
998	\\ DetDepthRef (bool) If True then refine DetDepth during calibration/recalibration calculation.
999	\\ distance (float) The distance (mm) from sample to detector plane.
1000	\\ ellipses (list:lists) Each object in ellipses is a list [center,phi,radii,color] where
1001	center (list) is location (mm) of the ellipse center on the detector plane, phi is the
1002	rotation of the ellipse minor axis from the x-axis, and radii are the minor & major
1003	radii of the ellipse. If radii[0] is negative then parameters describe a hyperbola. Color
1004	is the selected drawing color (one of 'b', 'g' ,'r') for the ellipse/hyperbola.
1005	\\ edgemin (float) Not used; parameter in EdgeFinder code.
1006	\\ fullIntegrate (bool) If True then integrate over full 360 deg azimuthal range.
1007	\\ GonioAngles (list:floats) The 'Omega','Chi','Phi' goniometer angles used for this image.
1008	Required for texture calculations.
1009	\\ invert_x (bool) If True display the image with the x-axis inverted.
1010	\\ invert_y (bool) If True display the image with the y-axis inverted.
1011	\\ IOtth (list:floats) The minimum and maximum 2-theta values to be used for integration.
1012	\\ LRazimuth (list:floats) The minimum and maximum azimuth values to be used for integration.
1013	\\ Oblique (list:float,bool) If True apply a detector absorption correction using the value to the
1014	intensities obtained during integration.
1015	\\ outAzimuths (int) The number of azimuth pie slices.
1016	\\ outChannels (int) The number of 2-theta steps.
1017	\\ pixelSize (list:ints) The X,Y dimensions (microns) of each pixel.
1018	\\ pixLimit (int) A box in the image with 2*pixLimit+1 edges is searched to find the maximum.
1019	This value (I) along with the minimum (Ib) in the box is reported by :func:`GSASIIimage.ImageLocalMax`
1020	and subject to cutoff in :func:`GSASIIimage.makeRing`.
1021	Locations are used to construct rings of points for calibration calcualtions.
1022	\\ PolaVal (list:float,bool) If type='SASD' and if True, apply polarization correction to intensities from
1023	integration using value.
1024	\\ rings (list:lists) Each entry is [X,Y,dsp] where X & Y are lists of x,y coordinates around a
1025	diffraction ring with the same d-spacing (dsp)
1026	\\ ring (list) The x,y coordinates of the >5 points on an inner ring
1027	selected by the user,
1028	\\ Range (list) The minimum & maximum values of the image
1029	\\ rotation (float) The angle between the x-axis and the vector about which the
1030	detector is tilted. Constrained to -180 to 180 deg.
1031	\\ SampleShape (str) Currently only 'Cylinder'. Sample shape for Debye-Scherrer experiments; used for absorption
1032	calculations.
1033	\\ SampleAbs (list: float,bool) Value of absorption coefficient for Debye-Scherrer experimnents, flag if True
1034	to cause correction to be applied.
1035	\\ setDefault (bool) If True the use the image controls values for all new images to be read. (might be removed)
1036	\\ setRings (bool) If True then display all the selected x,y ring positions (vida supra rings) used in the calibration.
1037	\\ showLines (bool) If True then isplay the integration limits to be used.
1038	\\ size (list:int) The number of pixels on the image x & y axes
1039	\\ type (str) One of 'PWDR', 'SASD' or 'REFL' for powder, small angle or reflectometry data, respectively.
1040	\\ tilt (float) The angle the detector normal makes with the incident beam; range -90 to 90.
1041	\\ wavelength (float) The radiation wavelength (:math:`\\AA`) as entered by the user
1042	(or someday obtained from the image header).
1043	Masks Arcs (list: lists) Each entry [2-theta,[azimuth[0],azimuth[1]],thickness] describes an arc mask
1044	to be excluded from integration
1045	\\ Frames (list:lists) Each entry describes the x,y points (3 or more - mm) that describe a frame outside
1046	of which is excluded from recalibration and integration. Only one frame is allowed.
1047	\\ Points (list:lists) Each entry [x,y,radius] (mm) describes an excluded spot on the image to be excluded
1048	from integration.
1049	\\ Polygons (list:lists) Each entry is a list of 3+ [x,y] points (mm) that describe a polygon on the image
1050	to be excluded from integration.
1051	\\ Rings (list: lists) Each entry [2-theta,thickness] describes a ring mask
1052	to be excluded from integration.
1053	\\ Thresholds (list:[tuple,list]) [(Imin,Imax),[Imin,Imax]] This gives lower and upper limits for points on the image to be included
1054	in integrsation. The tuple is the image intensity limits and the list are those set by the user.
1055	\\ SpotMask (dict: int & array)
1056	'esdMul'(int) number of standard deviations above mean ring intensity to mask
1057	'spotMask' (bool array) the spot mask for every pixel in image
1058
1059	Stress/Strain Sample phi (float) Sample rotation about vertical axis.
1060	\\ Sample z (float) Sample translation from the calibration sample position (for Sample phi = 0)
1061	These will be restricted by space group symmetry; result of strain fit refinement.
1062	\\ Type (str) 'True' or 'Conventional': The strain model used for the calculation.
1063	\\ d-zero (list:dict) Each item is for a diffraction ring on the image; all items are from the same phase
1064	and are used to determine the strain tensor.
1065	The dictionary items are:
1066	'Dset': (float) True d-spacing for the diffraction ring; entered by the user.
1067	'Dcalc': (float) Average calculated d-spacing determined from strain coeff.
1068	'Emat': (list: float) The strain tensor elements e11, e12 & e22 (e21=e12, rest are 0)
1069	'Esig': (list: float) Esds for Emat from fitting.
1070	'pixLimit': (int) Search range to find highest point on ring for each data point
1071	'cutoff': (float) I/Ib cutoff for searching.
1072	'ImxyObs': (list: lists) [[X],[Y]] observed points to be used for strain calculations.
1073	'ImtaObs': (list: lists) [[d],[azm]] transformed via detector calibration from ImxyObs.
1074	'ImtaCalc': (list: lists [[d],[azm]] calculated d-spacing & azimuth from fit.
1075
1076	====================== ====================== ====================================================
1077
1078	.. _parmDict_table:
1079
1080	.. index::
1081	single: Parameter dictionary
1082
1083	Parameter Dictionary
1084	-------------------------
1085
1086	The parameter dictionary contains all of the variable parameters for the refinement.
1087	The dictionary keys are the name of the parameter (<phase>:<hist>:<name>:<atom>).
1088	It is prepared in two ways. When loaded from the tree
1089	(in :meth:`GSASIIdataGUI.GSASII.MakeLSParmDict` and
1090	:meth:`GSASIIIO.ExportBaseclass.loadParmDict`),
1091	the values are lists with two elements: ``[value, refine flag]``
1092
1093	When loaded from the GPX file (in
1094	:func:`GSASIIstrMain.Refine` and :func:`GSASIIstrMain.SeqRefine`), the value in the
1095	dict is the actual parameter value (usually a float, but sometimes a
1096	letter or string flag value (such as I or A for iso/anisotropic).
1097
1098	Texture implementation
1099	------------------------------
1100
1101	There are two different places where texture can be treated in GSAS-II.
1102	One is for mitigating the effects of texture in a structural refinement.
1103	The other is for texture characterization.
1104
1105	For reducing the effect of texture in a structural refinement
1106	there are entries labeled preferred orientation in each phase's
1107	data tab. Two different approaches can be used for this, the March-Dollase
1108	model and spherical harmonics.
1109
1110	For the March-Dollase model, one axis in reciprocal space is designated as
1111	unique (defaulting to the 001 axis) and reflections are corrected
1112	according to the angle they make with this axis depending on
1113	the March-Dollase ratio. (If unity, no correction is made).
1114	The ratio can be greater than one or less than one depending on if
1115	crystallites oriented along the designated axis are
1116	overrepresented or underrepresented. For most crystal systems there is an
1117	obvious choice for the direction of the unique axis and then only a single
1118	term needs to be refined. If the number is close to 1, then the correction
1119	is not needed.
1120
1121	The second method for reducing the effect of texture in a structural
1122	refinement is to create a crystallite orientation probability surface as an
1123	expansion in terms spherical harmonic functions. Only functions consistent with
1124	cylindrical diffraction suymmetry and having texture symmetry
1125	consistent with the Laue class of phase are used and are allowed,
1126	so the higher the symmetry the fewer terms that are available for a given spherical harmonics order.
1127	To use this correction, select the lowest order that provides
1128	refinable terms and perform a refinement. If the texture index remains close to
1129	one, then the correction is not needed. If a significant improvement is
1130	noted in the profile Rwp, one may wish to see if a higher order expansion
1131	gives an even larger improvement.
1132
1133	To characterize texture in a material, generally one needs data collected with the
1134	sample at multiple orientations or, for TOF, with detectors at multiple
1135	locations around the sample. In this case the detector orientation is given in
1136	each histogram's Sample Parameters and the sample's orientation is described
1137	with the Euler angles specifed on the phase's Texture tab, which is also
1138	where the texture type (cylindrical, rolling,...) and the spherical
1139	harmonic order is selected. This should not be used with a single dataset and
1140	should not be used if the preferred orientations corrections are used.
1141
1142	The coordinate system used for texture characterization is defined where
1143	the sample coordinates (Psi, gamma) are defined with an instrument coordinate
1144	system (I, J, K) such that K is normal to the diffraction plane and J is coincident with the
1145	direction of the incident radiation beam toward the source. We further define
1146	a standard set of right-handed goniometer eulerian angles (Omega, Chi, Phi) so that Omega and Phi are
1147	rotations about K and Chi is a rotation about J when Omega = 0. Finally, as the sample
1148	may be mounted so that the sample coordinate system (Is, Js, Ks) does not coincide with
1149	the instrument coordinate system (I, J, K), we define three eulerian sample rotation angles
1150	(Omega-s, Chi-s, Phi-s) that describe the rotation from (Is, Js, Ks) to (I, J, K). The sample rotation
1151	angles are defined so that with the goniometer angles at zero Omega-s and Phi-s are rotations
1152	about K and Chi-s is a rotation about J.
1153
1154	Three typical examples:
1155
1156	1) Bragg-Brentano laboratory diffractometer: Chi=0
1157	2) Debye-Scherrer counter detector; sample capillary axis perpendicular to diffraction plane: Chi=90
1158	3) Debye-Scherrer 2D area detector positioned directly behind sample; sample capillary axis horizontal; Chi=0
1159
1160	NB: The area detector azimuthal angle will equal 0 in horizontal plane to right as viewed from x-ray source and will equal
1161	90 at vertical "up" direction.
1162
1163	ISODISTORT implementation
1164	------------------------------
1165
1166	CIFs prepared with the ISODISTORT web site
1167	https://stokes.byu.edu/iso/isodistort_version5.6.1/isodistort.php
1168	[B. J. Campbell, H. T. Stokes, D. E. Tanner, and D. M. Hatch, "ISODISPLACE: An Internet Tool for Exploring Structural Distortions."
1169	J. Appl. Cryst. 39, 607-614 (2006).] can be read into GSAS-II using import CIF. This will cause constraints to be established for
1170	structural distortion modes read from the CIF. At present, of the five types of modes only displacive(``_iso_displacivemode``...)
1171	and occupancy (``_iso_occupancymode``...) are processed. Not yet processed: ``_iso_magneticmode``...,
1172	``_iso_rotationalmode``... & ``_iso_strainmode``...
1173
1174	The CIF importer :mod:`G2phase_CIF` implements class :class:`G2phase_CIF.CIFPhaseReader` which offers two methods associated
1175	with ISODISTORT (ID) input. Method :meth:`G2phase_CIF.CIFPhaseReader.ISODISTORT_test` checks to see if a CIF block contains
1176	the loops with ``_iso_displacivemode_label`` or ``_iso_occupancymode_label`` items. If so, method
1177	:meth:`G2phase_CIF.CIFPhaseReader.ISODISTORT_proc` is called to read and interpret them. The results are placed into the
1178	reader object's ``.Phase`` class variable as a dict item with key ``'ISODISTORT'``.
1179
1180	Note that each mode ID has a long label with a name such as Pm-3m[1/2,1/2,1/2]R5+(a,a,0)[La:b:dsp]T1u(a). Function
1181	:func:`G2phase_CIF.ISODISTORT_shortLbl` is used to create a short name for this, such as R5_T1u(a) which is made unique
1182	by addition of _n if the short name is duplicated. As each mode is processed, a constraint corresponding to that mode is
1183	created and is added to list in the reader object's ``.Constraints`` class variable. Items placed into that list can either
1184	be a list, which corresponds to a function (new var) type :ref:`constraint definition <Constraints_table>` entry, or an item
1185	can be a dict, which provides help information for each constraint.
1186
1187	------------------------------
1188	Displacive modes
1189	------------------------------
1190
1191	The coordinate variables, as named by ISODISTORT, are placed in ``.Phase['ISODISTORT']['IsoVarList']`` and the
1192	corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed in ``.Phase['ISODISTORT']['G2VarList']``.
1193	The mode variables, as named by ISODISTORT, are placed in ``.Phase['ISODISTORT']['IsoModeList']`` and the
1194	corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed in ``.Phase['ISODISTORT']['G2ModeList']``.
1195	[Use ``str(G2VarObj)`` to get the variable name from the G2VarObj object, but note that the phase number, n, for the prefix
1196	"n::" cannot be determined as the phase number is not yet assigned.]
1197
1198	Displacive modes are a bit complex in that they relate to delta displacements, relative to an offset value for each coordinate,
1199	and because the modes are normalized. While GSAS-II also uses displacements, these are added to the coordinates after
1200	each refinement cycle and then the delta values are set to zero.
1201	ISODISTORT uses fixed offsets (subtracted from the actual position
1202	to obtain the delta values) that are taken from the parent structure coordinate and the initial offset value
1203	(in ``_iso_deltacoordinate_value``) and these are placed in
1204	``.Phase['ISODISTORT']['G2coordOffset']`` in the same order as ``.Phase['ISODISTORT']['G2ModeList']``,
1205	``.Phase['ISODISTORT']['IsoVarList']`` and ''.Phase[ISODISTORT']['G2parentCoords']''.'
1206
1207	The normalization factors (which the delta values are divided by)
1208	are taken from ``_iso_displacivemodenorm_value`` and are placed in ``.Phase['ISODISTORT']['NormList']`` in the same
1209	order as as ``...['IsoModeList']`` and ``...['G2ModeList']``.
1210
1211	The CIF contains a sparse matrix, from the ``loop_`` containing ``_iso_displacivemodematrix_value`` which provides the equations
1212	for determining the mode values from the coordinates, that matrix is placed in ``.Phase['ISODISTORT']['Mode2VarMatrix']``.
1213	The matrix is inverted to produce ``.Phase['ISODISTORT']['Var2ModeMatrix']``, which determines how to compute the
1214	mode values from the delta coordinate values. These values are used for the in :func:`GSASIIconstrGUI.ShowIsoDistortCalc`,
1215	which shows coordinate and mode values, the latter with s.u. values.
1216
1217	------------------------------
1218	Occupancy modes
1219	------------------------------
1220
1221
1222	The delta occupancy variables, as named by ISODISTORT, are placed in
1223	``.Phase['ISODISTORT']['OccVarList']`` and the corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed
1224	in ``.Phase['ISODISTORT']['G2OccVarList']``. The mode variables, as named by ISODISTORT, are placed in
1225	``.Phase['ISODISTORT']['OccModeList']`` and the corresponding :class:`GSASIIobj.G2VarObj` objects for each are placed
1226	in ``.Phase['ISODISTORT']['G2OccModeList']``.
1227
1228	Occupancy modes, like Displacive modes, are also refined as delta values. However, GSAS-II directly refines the fractional
1229	occupancies. Offset values for each atom, are taken from ``_iso_occupancy_formula`` and are placed in
1230	``.Phase['ISODISTORT']['ParentOcc]``. (Offset values are subtracted from the actual position to obtain the delta values.)
1231	Modes are normalized (where the mode values are divided by the normalization factor) are taken from ``_iso_occupancymodenorm_value``
1232	and are placed in ``.Phase['ISODISTORT']['OccNormList']`` in the same order as as ``...['OccModeList']`` and
1233	``...['G2OccModeList']``.
1234
1235	The CIF contains a sparse matrix, from the ``loop_`` containing ``_iso_occupancymodematrix_value``, which provides the
1236	equations for determining the mode values from the coordinates. That matrix is placed in ``.Phase['ISODISTORT']['Occ2VarMatrix']``.
1237	The matrix is inverted to produce ``.Phase['ISODISTORT']['Var2OccMatrix']``, which determines how to compute the
1238	mode values from the delta coordinate values.
1239
1240
1241	------------------------------
1242	Mode Computations
1243	------------------------------
1244
1245	Constraints are processed after the CIF has been read in :meth:`GSASIIdataGUI.GSASII.OnImportPhase` or
1246	:meth:`GSASIIscriptable.G2Project.add_phase` by moving them from the reader object's ``.Constraints``
1247	class variable to the Constraints tree entry's ['Phase'] list (for list items defining constraints) or
1248	the Constraints tree entry's ['_Explain'] dict (for dict items defining constraint help information)
1249
1250	The information in ``.Phase['ISODISTORT']`` is used in :func:`GSASIIconstrGUI.ShowIsoDistortCalc` which shows coordinate and mode
1251	values, the latter with s.u. values. This can be called from the Constraints and Phase/Atoms tree items.
1252
1253	Before each refinement, constraints are processed as :ref:`described elsewhere <Constraints_processing>`. After a refinement
1254	is complete, :func:`GSASIImapvars.PrintIndependentVars` shows the shifts and s.u.'s on the refined modes,
1255	using GSAS-II values, but :func:`GSASIIstrIO.PrintISOmodes` prints the ISODISTORT modes as computed in the web site.
1256
1257
1258	.. _ParameterLimits:
1259
1260	.. index::
1261	single: Parameter limits
1262
1263	Parameter Limits
1264	------------------------------
1265
1266	One of the most often requested "enhancements" for GSAS-II would be the inclusion
1267	of constraints to force parameters such as occupancies or Uiso values to stay within
1268	expected ranges. While it is possible for users to supply their own restraints that would
1269	perform this by supplying an appropriate expression with the "General" restraints, the
1270	GSAS-II authors do not feel that use of restraints or constraints are a good solution for
1271	this common problem where parameters refine to non-physical values. This is because when
1272	this occurs, most likely one of the following cases is occurring:
1273
1274	#. there is a significant problem
1275	with the model, for example for an x-ray fit if an O atom is placed where a S is actually
1276	present, the Uiso will refine artificially small or the occupancy much larger than unity
1277	to try to compensate for the missing electrons; or
1278
1279	#. the data are simply insensitive
1280	to the parameter or combination of parameters, for example unless very high-Q data
1281	are included, the effects of a occupancy and Uiso value can have compensating effects,
1282	so an assumption must be made; likewise, with neutron data natural-abundance V atoms
1283	are nearly invisible due to weak coherent scattering. No parameters can be fit for a
1284	V atom with neutrons.
1285
1286	#. the parameter is non-physical (such as a negative Uiso value) but within
1287	two sigma (sigma = standard uncertainty, aka e.s.d.) of a reasonable value,
1288	in which case the
1289	value is not problematic as it is experimentally indistinguishable from an
1290	expected value.
1291
1292	#. there is a systematic problem with the data (experimental error)
1293
1294	In all these cases, this situation needs to be reviewed by a crystallographer to decide
1295	how to best determine a structural model for these data. An implementation with a constraint
1296	or restraint is likely to simply hide the problem from the user, making it more probable
1297	that a poor model choice is obtained.
1298
1299	What GSAS-II does implement is to allow users to specify ranges for parameters
1300	that works by disabling
1301	refinement of parameters that refine beyond either a lower limit or an upper limit, where
1302	either or both may be optionally specified. Parameters limits are specified in the Controls
1303	tree entry in dicts named as ``Controls['parmMaxDict']`` and ``Controls['parmMinDict']``, where
1304	the keys are :class:`G2VarObj` objects corresponding to standard GSAS-II variable
1305	(see :func:`getVarDescr` and :func:`CompileVarDesc`) names, where a
1306	wildcard ('*') may optionally be used for histogram number or atom number
1307	(phase number is intentionally not allowed as a wildcard as it makes little sense
1308	to group the same parameter together different phases). Note
1309	that :func:`prmLookup` is used to see if a name matches a wildcard. The upper or lower limit
1310	is placed into these dicts as a float value. These values can be edited using the window
1311	created by the Calculate/"View LS parms" menu command or in scripting with the
1312	:meth:`GSASIIscriptable.G2Project.set_Controls` function.
1313	In the GUI, a checkbox labeled "match all histograms/atoms" is used to insert a wildcard
1314	into the appropriate part of the variable name.
1315
1316	When a refinement is conducted, routine :func:`GSASIIstrMain.dropOOBvars` is used to
1317	find parameters that have refined to values outside their limits. If this occurs, the parameter
1318	is set to the limiting value and the variable name is added to a list of frozen variables
1319	(as a :class:`G2VarObj` objects) kept in a list in the
1320	``Controls['parmFrozen']`` dict. In a sequential refinement, this is kept separate for
1321	each histogram as a list in
1322	``Controls['parmFrozen'][histogram]`` (where the key is the histogram name) or as a list in
1323	``Controls['parmFrozen']['FrozenList']`` for a non-sequential fit.
1324	This allows different variables
1325	to be frozen in each section of a sequential fit.
1326	Frozen parameters are not included in refinements through removal from the
1327	list of parameters to be refined (``varyList``) in :func:`GSASIIstrMain.Refine` or
1328	:func:`GSASIIstrMain.SeqRefine`.
1329	The data window for the Controls tree item shows the number of Frozen variables and
1330	the individual variables can be viewed with the Calculate/"View LS parms" menu window or
1331	obtained with :meth:`GSASIIscriptable.G2Project.get_Frozen`.
1332	Once a variable is frozen, it will not be refined in any
1333	future refinements unless the the variable is removed (manually) from the list. This can also
1334	be done with the Calculate/"View LS parms" menu window or
1335	:meth:`GSASIIscriptable.G2Project.set_Frozen`.
1336
1337
1338	.. seealso::
1339	:class:`G2VarObj`
1340	:func:`getVarDescr`
1341	:func:`CompileVarDesc`
1342	:func:`prmLookup`
1343	:class:`GSASIIctrlGUI.ShowLSParms`
1344	:class:`GSASIIctrlGUI.VirtualVarBox`
1345	:func:`GSASIIstrIO.SetUsedHistogramsAndPhases`
1346	:func:`GSASIIstrIO.SaveUpdatedHistogramsAndPhases`
1347	:func:`GSASIIstrIO.SetSeqResult`
1348	:func:`GSASIIstrMain.dropOOBvars`
1349	:meth:`GSASIIscriptable.G2Project.set_Controls`
1350	:meth:`GSASIIscriptable.G2Project.get_Frozen`
1351	:meth:`GSASIIscriptable.G2Project.set_Frozen`
1352
1353	Classes and routines
1354	----------------------
1355
1356	'''
1357	from __future__ import division, print_function
1358	import platform
1359	import re
1360	import random as ran
1361	import sys
1362	import os.path
1363	if '2' in platform.python_version_tuple()[0]:
1364	import cPickle
1365	else:
1366	import pickle as cPickle
1367	import GSASIIpath
1368	import GSASIImath as G2mth
1369	import GSASIIspc as G2spc
1370	import numpy as np
1371
1372	GSASIIpath.SetVersionNumber("$Revision: 5148 $")
1373
1374	DefaultControls = {
1375	'deriv type':'analytic Hessian',
1376	'min dM/M':0.001,'shift factor':1.,'max cyc':3,'F**2':False,'SVDtol':1.e-6,
1377	'UsrReject':{'minF/sig':0.,'MinExt':0.01,'MaxDF/F':100.,'MaxD':500.,'MinD':0.05},
1378	'Copy2Next':False,'Reverse Seq':False,'HatomFix':False,
1379	'Author':'no name','newLeBail':False,
1380	'FreePrm1':'Sample humidity (%)',
1381	'FreePrm2':'Sample voltage (V)',
1382	'FreePrm3':'Applied load (MN)',
1383	'ShowCell':False,
1384	}
1385	'''Values to be used as defaults for the initial contents of the ``Controls``
1386	data tree item.
1387	'''
1388	def StripUnicode(string,subs='.'):
1389	'''Strip non-ASCII characters from strings
1390
1391	:param str string: string to strip Unicode characters from
1392	:param str subs: character(s) to place into string in place of each
1393	Unicode character. Defaults to '.'
1394
1395	:returns: a new string with only ASCII characters
1396	'''
1397	s = ''
1398	for c in string:
1399	if ord(c) < 128:
1400	s += c
1401	else:
1402	s += subs
1403	return s
1404	# return s.encode('ascii','replace')
1405
1406	def MakeUniqueLabel(lbl,labellist):
1407	'''Make sure that every a label is unique against a list by adding
1408	digits at the end until it is not found in list.
1409
1410	:param str lbl: the input label
1411	:param list labellist: the labels that have already been encountered
1412	:returns: lbl if not found in labellist or lbl with ``_1-9`` (or
1413	``_10-99``, etc.) appended at the end
1414	'''
1415	lbl = StripUnicode(lbl.strip(),'_')
1416	if not lbl: # deal with a blank label
1417	lbl = '_1'
1418	if lbl not in labellist:
1419	labellist.append(lbl)
1420	return lbl
1421	i = 1
1422	prefix = lbl
1423	if '_' in lbl:
1424	prefix = lbl[:lbl.rfind('_')]
1425	suffix = lbl[lbl.rfind('_')+1:]
1426	try:
1427	i = int(suffix)+1
1428	except: # suffix could not be parsed
1429	i = 1
1430	prefix = lbl
1431	while prefix+'_'+str(i) in labellist:
1432	i += 1
1433	else:
1434	lbl = prefix+'_'+str(i)
1435	labellist.append(lbl)
1436	return lbl
1437
1438	PhaseIdLookup = {}
1439	'''dict listing phase name and random Id keyed by sequential phase index as a str;
1440	best to access this using :func:`LookupPhaseName`
1441	'''
1442	PhaseRanIdLookup = {}
1443	'''dict listing phase sequential index keyed by phase random Id;
1444	best to access this using :func:`LookupPhaseId`
1445	'''
1446	HistIdLookup = {}
1447	'''dict listing histogram name and random Id, keyed by sequential histogram index as a str;
1448	best to access this using :func:`LookupHistName`
1449	'''
1450	HistRanIdLookup = {}
1451	'''dict listing histogram sequential index keyed by histogram random Id;
1452	best to access this using :func:`LookupHistId`
1453	'''
1454	AtomIdLookup = {}
1455	'''dict listing for each phase index as a str, the atom label and atom random Id,
1456	keyed by atom sequential index as a str;
1457	best to access this using :func:`LookupAtomLabel`
1458	'''
1459	AtomRanIdLookup = {}
1460	'''dict listing for each phase the atom sequential index keyed by atom random Id;
1461	best to access this using :func:`LookupAtomId`
1462	'''
1463	ShortPhaseNames = {}
1464	'''a dict containing a possibly shortened and when non-unique numbered
1465	version of the phase name. Keyed by the phase sequential index.
1466	'''
1467	ShortHistNames = {}
1468	'''a dict containing a possibly shortened and when non-unique numbered
1469	version of the histogram name. Keyed by the histogram sequential index.
1470	'''
1471
1472	#VarDesc = {} # removed 1/30/19 BHT as no longer needed (I think)
1473	#''' This dictionary lists descriptions for GSAS-II variables,
1474	#as set in :func:`CompileVarDesc`. See that function for a description
1475	#for how keys and values are written.
1476	#'''
1477
1478	reVarDesc = {}
1479	''' This dictionary lists descriptions for GSAS-II variables where
1480	keys are compiled regular expressions that will match the name portion
1481	of a parameter name. Initialized in :func:`CompileVarDesc`.
1482	'''
1483
1484	reVarStep = {}
1485	''' This dictionary lists the preferred step size for numerical
1486	derivative computation w/r to a GSAS-II variable. Keys are compiled
1487	regular expressions and values are the step size for that parameter.
1488	Initialized in :func:`CompileVarDesc`.
1489	'''
1490	# create a default space group object for P1; N.B. fails when building documentation
1491	try:
1492	P1SGData = G2spc.SpcGroup('P 1')[1] # data structure for default space group
1493	except:
1494	pass
1495
1496	def GetPhaseNames(fl):
1497	''' Returns a list of phase names found under 'Phases' in GSASII gpx file
1498	NB: there is another one of these in GSASIIstrIO.py that uses the gpx filename
1499
1500	:param file fl: opened .gpx file
1501	:return: list of phase names
1502	'''
1503	PhaseNames = []
1504	while True:
1505	try:
1506	data = cPickle.load(fl)
1507	except EOFError:
1508	break
1509	datum = data[0]
1510	if 'Phases' == datum[0]:
1511	for datus in data[1:]:
1512	PhaseNames.append(datus[0])
1513	fl.seek(0) #reposition file
1514	return PhaseNames
1515
1516	def SetNewPhase(Name='New Phase',SGData=None,cell=None,Super=None):
1517	'''Create a new phase dict with default values for various parameters
1518
1519	:param str Name: Name for new Phase
1520
1521	:param dict SGData: space group data from :func:`GSASIIspc:SpcGroup`;
1522	defaults to data for P 1
1523
1524	:param list cell: unit cell parameter list; defaults to
1525	[1.0,1.0,1.0,90.,90,90.,1.]
1526
1527	'''
1528	if SGData is None: SGData = P1SGData
1529	if cell is None: cell=[1.0,1.0,1.0,90.,90.,90.,1.]
1530	phaseData = {
1531	'ranId':ran.randint(0,sys.maxsize),
1532	'General':{
1533	'Name':Name,
1534	'Type':'nuclear',
1535	'Modulated':False,
1536	'AtomPtrs':[3,1,7,9],
1537	'SGData':SGData,
1538	'Cell':[False,]+cell,
1539	'Pawley dmin':1.0,
1540	'Data plot type':'None',
1541	'SH Texture':{
1542	'Order':0,
1543	'Model':'cylindrical',
1544	'Sample omega':[False,0.0],
1545	'Sample chi':[False,0.0],
1546	'Sample phi':[False,0.0],
1547	'SH Coeff':[False,{}],
1548	'SHShow':False,
1549	'PFhkl':[0,0,1],
1550	'PFxyz':[0,0,1],
1551	'PlotType':'Pole figure',
1552	'Penalty':[['',],0.1,False,1.0]}},
1553	'Atoms':[],
1554	'Drawing':{},
1555	'Histograms':{},
1556	'Pawley ref':[],
1557	'RBModels':{},
1558	}
1559	if Super and Super.get('Use',False):
1560	phaseData['General'].update({'Modulated':True,'Super':True,'SuperSg':Super['ssSymb']})
1561	phaseData['General']['SSGData'] = G2spc.SSpcGroup(SGData,Super['ssSymb'])[1]
1562	phaseData['General']['SuperVec'] = [Super['ModVec'],False,Super['maxH']]
1563
1564	return phaseData
1565
1566	def ReadCIF(URLorFile):
1567	'''Open a CIF, which may be specified as a file name or as a URL using PyCifRW
1568	(from James Hester).
1569	The open routine gets confused with DOS names that begin with a letter and colon
1570	"C:\\dir\" so this routine will try to open the passed name as a file and if that
1571	fails, try it as a URL
1572
1573	:param str URLorFile: string containing a URL or a file name. Code will try first
1574	to open it as a file and then as a URL.
1575
1576	:returns: a PyCifRW CIF object.
1577	'''
1578	import CifFile as cif # PyCifRW from James Hester
1579
1580	# alternate approach:
1581	#import urllib
1582	#ciffile = 'file:'+urllib.pathname2url(filename)
1583
1584	try:
1585	fp = open(URLorFile,'r')
1586	cf = cif.ReadCif(fp)
1587	fp.close()
1588	return cf
1589	except IOError:
1590	return cif.ReadCif(URLorFile)
1591
1592	def TestIndexAll():
1593	'''Test if :func:`IndexAllIds` has been called to index all phases and
1594	histograms (this is needed before :func:`G2VarObj` can be used.
1595
1596	:returns: Returns True if indexing is needed.
1597	'''
1598	if PhaseIdLookup or AtomIdLookup or HistIdLookup:
1599	return False
1600	return True
1601
1602	def IndexAllIds(Histograms,Phases):
1603	'''Scan through the used phases & histograms and create an index
1604	to the random numbers of phases, histograms and atoms. While doing this,
1605	confirm that assigned random numbers are unique -- just in case lightning
1606	strikes twice in the same place.
1607
1608	Note: this code assumes that the atom random Id (ranId) is the last
1609	element each atom record.
1610
1611	This is called when phases & histograms are looked up
1612	in these places (only):
1613
1614	* :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` (which loads the histograms and phases from a GPX file),
1615	* :meth:`~GSASIIdataGUI.GSASII.GetUsedHistogramsAndPhasesfromTree` (which does the same thing but from the data tree.)
1616	* :meth:`~GSASIIdataGUI.GSASII.OnFileClose` (clears out an old project)
1617
1618	Note that globals :data:`PhaseIdLookup` and :data:`PhaseRanIdLookup` are
1619	also set in :func:`AddPhase2Index` to temporarily assign a phase number
1620	as a phase is being imported.
1621
1622	TODO: do we need a lookup for rigid body variables?
1623	'''
1624	# process phases and atoms
1625	PhaseIdLookup.clear()
1626	PhaseRanIdLookup.clear()
1627	AtomIdLookup.clear()
1628	AtomRanIdLookup.clear()
1629	ShortPhaseNames.clear()
1630	for ph in Phases:
1631	cx,ct,cs,cia = Phases[ph]['General']['AtomPtrs']
1632	ranId = Phases[ph]['ranId']
1633	while ranId in PhaseRanIdLookup:
1634	# Found duplicate random Id! note and reassign
1635	print ("\n\n*** Phase "+str(ph)+" has repeated ranId. Fixing.\n")
1636	Phases[ph]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1637	pId = str(Phases[ph]['pId'])
1638	PhaseIdLookup[pId] = (ph,ranId)
1639	PhaseRanIdLookup[ranId] = pId
1640	shortname = ph #[:10]
1641	while shortname in ShortPhaseNames.values():
1642	shortname = ph[:8] + ' ('+ pId + ')'
1643	ShortPhaseNames[pId] = shortname
1644	AtomIdLookup[pId] = {}
1645	AtomRanIdLookup[pId] = {}
1646	for iatm,at in enumerate(Phases[ph]['Atoms']):
1647	ranId = at[cia+8]
1648	while ranId in AtomRanIdLookup[pId]: # check for dups
1649	print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1650	at[cia+8] = ranId = ran.randint(0,sys.maxsize)
1651	AtomRanIdLookup[pId][ranId] = str(iatm)
1652	if Phases[ph]['General']['Type'] == 'macromolecular':
1653	label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1654	else:
1655	label = at[ct-1]
1656	AtomIdLookup[pId][str(iatm)] = (label,ranId)
1657	# process histograms
1658	HistIdLookup.clear()
1659	HistRanIdLookup.clear()
1660	ShortHistNames.clear()
1661	for hist in Histograms:
1662	ranId = Histograms[hist]['ranId']
1663	while ranId in HistRanIdLookup:
1664	# Found duplicate random Id! note and reassign
1665	print ("\n\n*** Histogram "+str(hist)+" has repeated ranId. Fixing.\n")
1666	Histograms[hist]['ranId'] = ranId = ran.randint(0,sys.maxsize)
1667	hId = str(Histograms[hist]['hId'])
1668	HistIdLookup[hId] = (hist,ranId)
1669	HistRanIdLookup[ranId] = hId
1670	shortname = hist[:15]
1671	while shortname in ShortHistNames.values():
1672	shortname = hist[:11] + ' ('+ hId + ')'
1673	ShortHistNames[hId] = shortname
1674
1675	def AddPhase2Index(rdObj,filename):
1676	'''Add a phase to the index during reading
1677	Used where constraints are generated during import (ISODISTORT CIFs)
1678	'''
1679	ranId = rdObj.Phase['ranId']
1680	ph = 'from '+filename # phase is not named yet
1681	if ranId in PhaseRanIdLookup: return
1682	maxph = -1
1683	for r in PhaseRanIdLookup:
1684	maxph = max(maxph,int(PhaseRanIdLookup[r]))
1685	PhaseRanIdLookup[ranId] = pId = str(maxph + 1)
1686	PhaseIdLookup[pId] = (ph,ranId)
1687	shortname = 'from '+ os.path.splitext((os.path.split(filename))[1])[0]
1688	while shortname in ShortPhaseNames.values():
1689	shortname = ph[:8] + ' ('+ pId + ')'
1690	ShortPhaseNames[pId] = shortname
1691	AtomIdLookup[pId] = {}
1692	AtomRanIdLookup[pId] = {}
1693	for iatm,at in enumerate(rdObj.Phase['Atoms']):
1694	ranId = at[-1]
1695	while ranId in AtomRanIdLookup[pId]: # check for dups
1696	print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1697	at[-1] = ranId = ran.randint(0,sys.maxsize)
1698	AtomRanIdLookup[pId][ranId] = str(iatm)
1699	#if Phases[ph]['General']['Type'] == 'macromolecular':
1700	# label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1701	#else:
1702	# label = at[ct-1]
1703	label = at[0]
1704	AtomIdLookup[pId][str(iatm)] = (label,ranId)
1705
1706	def LookupAtomId(pId,ranId):
1707	'''Get the atom number from a phase and atom random Id
1708
1709	:param int/str pId: the sequential number of the phase
1710	:param int ranId: the random Id assigned to an atom
1711
1712	:returns: the index number of the atom (str)
1713	'''
1714	if not AtomRanIdLookup:
1715	raise Exception('Error: LookupAtomId called before IndexAllIds was run')
1716	if pId is None or pId == '':
1717	raise KeyError('Error: phase is invalid (None or blank)')
1718	pId = str(pId)
1719	if pId not in AtomRanIdLookup:
1720	raise KeyError('Error: LookupAtomId does not have phase '+pId)
1721	if ranId not in AtomRanIdLookup[pId]:
1722	raise KeyError('Error: LookupAtomId, ranId '+str(ranId)+' not in AtomRanIdLookup['+pId+']')
1723	return AtomRanIdLookup[pId][ranId]
1724
1725	def LookupAtomLabel(pId,index):
1726	'''Get the atom label from a phase and atom index number
1727
1728	:param int/str pId: the sequential number of the phase
1729	:param int index: the index of the atom in the list of atoms
1730
1731	:returns: the label for the atom (str) and the random Id of the atom (int)
1732	'''
1733	if not AtomIdLookup:
1734	raise Exception('Error: LookupAtomLabel called before IndexAllIds was run')
1735	if pId is None or pId == '':
1736	raise KeyError('Error: phase is invalid (None or blank)')
1737	pId = str(pId)
1738	if pId not in AtomIdLookup:
1739	raise KeyError('Error: LookupAtomLabel does not have phase '+pId)
1740	if index not in AtomIdLookup[pId]:
1741	raise KeyError('Error: LookupAtomLabel, ranId '+str(index)+' not in AtomRanIdLookup['+pId+']')
1742	return AtomIdLookup[pId][index]
1743
1744	def LookupPhaseId(ranId):
1745	'''Get the phase number and name from a phase random Id
1746
1747	:param int ranId: the random Id assigned to a phase
1748	:returns: the sequential Id (pId) number for the phase (str)
1749	'''
1750	if not PhaseRanIdLookup:
1751	raise Exception('Error: LookupPhaseId called before IndexAllIds was run')
1752	if ranId not in PhaseRanIdLookup:
1753	raise KeyError('Error: LookupPhaseId does not have ranId '+str(ranId))
1754	return PhaseRanIdLookup[ranId]
1755
1756	def LookupPhaseName(pId):
1757	'''Get the phase number and name from a phase Id
1758
1759	:param int/str pId: the sequential assigned to a phase
1760	:returns: (phase,ranId) where phase is the name of the phase (str)
1761	and ranId is the random # id for the phase (int)
1762	'''
1763	if not PhaseIdLookup:
1764	raise Exception('Error: LookupPhaseName called before IndexAllIds was run')
1765	if pId is None or pId == '':
1766	raise KeyError('Error: phase is invalid (None or blank)')
1767	pId = str(pId)
1768	if pId not in PhaseIdLookup:
1769	raise KeyError('Error: LookupPhaseName does not have index '+pId)
1770	return PhaseIdLookup[pId]
1771
1772	def LookupHistId(ranId):
1773	'''Get the histogram number and name from a histogram random Id
1774
1775	:param int ranId: the random Id assigned to a histogram
1776	:returns: the sequential Id (hId) number for the histogram (str)
1777	'''
1778	if not HistRanIdLookup:
1779	raise Exception('Error: LookupHistId called before IndexAllIds was run')
1780	if ranId not in HistRanIdLookup:
1781	raise KeyError('Error: LookupHistId does not have ranId '+str(ranId))
1782	return HistRanIdLookup[ranId]
1783
1784	def LookupHistName(hId):
1785	'''Get the histogram number and name from a histogram Id
1786
1787	:param int/str hId: the sequential assigned to a histogram
1788	:returns: (hist,ranId) where hist is the name of the histogram (str)
1789	and ranId is the random # id for the histogram (int)
1790	'''
1791	if not HistIdLookup:
1792	raise Exception('Error: LookupHistName called before IndexAllIds was run')
1793	if hId is None or hId == '':
1794	raise KeyError('Error: histogram is invalid (None or blank)')
1795	hId = str(hId)
1796	if hId not in HistIdLookup:
1797	raise KeyError('Error: LookupHistName does not have index '+hId)
1798	return HistIdLookup[hId]
1799
1800	def fmtVarDescr(varname):
1801	'''Return a string with a more complete description for a GSAS-II variable
1802
1803	:param str varname: A full G2 variable name with 2 or 3 or 4
1804	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1805
1806	:returns: a string with the description
1807	'''
1808	s,l = VarDescr(varname)
1809	return s+": "+l
1810
1811	def VarDescr(varname):
1812	'''Return two strings with a more complete description for a GSAS-II variable
1813
1814	:param str name: A full G2 variable name with 2 or 3 or 4
1815	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1816
1817	:returns: (loc,meaning) where loc describes what item the variable is mapped
1818	(phase, histogram, etc.) and meaning describes what the variable does.
1819	'''
1820
1821	# special handling for parameter names without a colon
1822	# for now, assume self-defining
1823	if varname.find(':') == -1:
1824	return "Global",varname
1825
1826	l = getVarDescr(varname)
1827	if not l:
1828	return ("invalid variable name ("+str(varname)+")!"),""
1829	# return "invalid variable name!",""
1830
1831	if not l[-1]:
1832	l[-1] = "(variable needs a definition! Set it in CompileVarDesc)"
1833
1834	if len(l) == 3: #SASD variable name!
1835	s = 'component:'+l[1]
1836	return s,l[-1]
1837	s = ""
1838	if l[0] is not None and l[1] is not None: # HAP: keep short
1839	if l[2] == "Scale": # fix up ambigous name
1840	l[5] = "Phase fraction"
1841	if l[0] == '*':
1842	lbl = 'Seq. ref.'
1843	else:
1844	lbl = ShortPhaseNames.get(l[0],'? #'+str(l[0]))
1845	if l[1] == '*':
1846	hlbl = 'Seq. ref.'
1847	else:
1848	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1849	if hlbl[:4] == 'HKLF':
1850	hlbl = 'Xtl='+hlbl[5:]
1851	elif hlbl[:4] == 'PWDR':
1852	hlbl = 'Pwd='+hlbl[5:]
1853	else:
1854	hlbl = 'Hist='+hlbl
1855	s = "Ph="+str(lbl)+" * "+str(hlbl)
1856	else:
1857	if l[2] == "Scale": # fix up ambigous name: must be scale factor, since not HAP
1858	l[5] = "Scale factor"
1859	if l[2] == 'Back': # background parameters are "special", alas
1860	s = 'Hist='+ShortHistNames.get(l[1],'? #'+str(l[1]))
1861	l[-1] += ' #'+str(l[3])
1862	elif l[4] is not None: # rigid body parameter or modulation parm
1863	lbl = ShortPhaseNames.get(l[0],'phase?')
1864	if 'RB' in l[2]: #rigid body parm
1865	s = "RB body #"+str(l[3])+" (type "+str(l[4])+") in "+str(lbl) + ','
1866	else: #modulation parm
1867	s = 'Atom %s wave %s in %s'%(LookupAtomLabel(l[0],l[3])[0],l[4],lbl)
1868	elif l[3] is not None: # atom parameter,
1869	lbl = ShortPhaseNames.get(l[0],'phase?')
1870	try:
1871	albl = LookupAtomLabel(l[0],l[3])[0]
1872	except KeyError:
1873	albl = 'Atom?'
1874	s = "Atom "+str(albl)+" in "+str(lbl)
1875	elif l[0] == '*':
1876	s = "All phases "
1877	elif l[0] is not None:
1878	lbl = ShortPhaseNames.get(l[0],'phase?')
1879	s = "Phase "+str(lbl)
1880	elif l[1] == '*':
1881	s = 'All hists'
1882	elif l[1] is not None:
1883	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1884	if hlbl[:4] == 'HKLF':
1885	hlbl = 'Xtl='+hlbl[5:]
1886	elif hlbl[:4] == 'PWDR':
1887	hlbl = 'Pwd='+hlbl[5:]
1888	else:
1889	hlbl = 'Hist='+hlbl
1890	s = str(hlbl)
1891	if not s:
1892	s = 'Global'
1893	return s,l[-1]
1894
1895	def getVarDescr(varname):
1896	'''Return a short description for a GSAS-II variable
1897
1898	:param str name: A full G2 variable name with 2 or 3 or 4
1899	colons (<p>:<h>:name[:<a1>][:<a2>])
1900
1901	:returns: a six element list as [`p`,`h`,`name`,`a1`,`a2`,`description`],
1902	where `p`, `h`, `a1`, `a2` are str values or `None`, for the phase number,
1903	the histogram number and the atom number; `name` will always be
1904	a str; and `description` is str or `None`.
1905	If the variable name is incorrectly formed (for example, wrong
1906	number of colons), `None` is returned instead of a list.
1907	'''
1908	l = varname.split(':')
1909	if len(l) == 2: #SASD parameter name
1910	return varname,l[0],getDescr(l[1])
1911	if len(l) == 3:
1912	l += [None,None]
1913	elif len(l) == 4:
1914	l += [None]
1915	elif len(l) != 5:
1916	return None
1917	for i in (0,1,3,4):
1918	if l[i] == "":
1919	l[i] = None
1920	l += [getDescr(l[2])]
1921	return l
1922
1923	def CompileVarDesc():
1924	'''Set the values in the variable lookup tables
1925	(:attr:`reVarDesc` and :attr:`reVarStep`).
1926	This is called in :func:`getDescr` and :func:`getVarStep` so this
1927	initialization is always done before use.
1928
1929	Note that keys may contain regular expressions, where '[xyz]'
1930	matches 'x' 'y' or 'z' (equivalently '[x-z]' describes this as range
1931	of values). '.*' matches any string. For example::
1932
1933	'AUiso':'Atomic isotropic displacement parameter',
1934
1935	will match variable ``'p::AUiso:a'``.
1936	If parentheses are used in the key, the contents of those parentheses can be
1937	used in the value, such as::
1938
1939	'AU([123][123])':'Atomic anisotropic displacement parameter U\\1',
1940
1941	will match ``AU11``, ``AU23``,... and `U11`, `U23` etc will be displayed
1942	in the value when used.
1943
1944	'''
1945	if reVarDesc: return # already done
1946	for key,value in {
1947	# derived or other sequential vars
1948	'([abc])$' : 'Lattice parameter, \\1, from Ai and Djk', # N.B. '$' prevents match if any characters follow
1949	u'\u03B1' : u'Lattice parameter, \u03B1, from Ai and Djk',
1950	u'\u03B2' : u'Lattice parameter, \u03B2, from Ai and Djk',
1951	u'\u03B3' : u'Lattice parameter, \u03B3, from Ai and Djk',
1952	# ambiguous, alas:
1953	'Scale' : 'Phase or Histogram scale factor',
1954	# Phase vars (p::<var>)
1955	'A([0-5])' : ('Reciprocal metric tensor component \\1',1e-5),
1956	'[vV]ol' : 'Unit cell volume', # probably an error that both upper and lower case are used
1957	# Atom vars (p::<var>:a)
1958	'dA([xyz])$' : ('change to atomic coordinate, \\1',1e-6),
1959	'A([xyz])$' : '\\1 fractional atomic coordinate',
1960	'AUiso':('Atomic isotropic displacement parameter',1e-4),
1961	'AU([123][123])':('Atomic anisotropic displacement parameter U\\1',1e-4),
1962	'Afrac': ('Atomic site fraction parameter',1e-5),
1963	'Amul': 'Atomic site multiplicity value',
1964	'AM([xyz])$' : 'Atomic magnetic moment parameter, \\1',
1965	# Hist (:h:<var>) & Phase (HAP) vars (p:h:<var>)
1966	'Back': 'Background term',
1967	'BkPkint;(.*)':'Background peak #\\1 intensity',
1968	'BkPkpos;(.*)':'Background peak #\\1 position',
1969	'BkPksig;(.*)':'Background peak #\\1 Gaussian width',
1970	'BkPkgam;(.*)':'Background peak #\\1 Cauchy width',
1971	# 'Back File' : 'Background file name',
1972	'BF mult' : 'Background file multiplier',
1973	'Bab([AU])': 'Babinet solvent scattering coef. \\1',
1974	'D([123][123])' : 'Anisotropic strain coef. \\1',
1975	'Extinction' : 'Extinction coef.',
1976	'MD' : 'March-Dollase coef.',
1977	'Mustrain;.*' : 'Microstrain coef.',
1978	'Size;.*' : 'Crystallite size value',
1979	'eA$' : 'Cubic mustrain value',
1980	'Ep$' : 'Primary extinction',
1981	'Es$' : 'Secondary type II extinction',
1982	'Eg$' : 'Secondary type I extinction',
1983	'Flack' : 'Flack parameter',
1984	'TwinFr' : 'Twin fraction',
1985	'Layer Disp' : 'Layer displacement along beam',
1986	#Histogram vars (:h:<var>)
1987	'Absorption' : 'Absorption coef.',
1988	'LayerDisp' : 'Bragg-Brentano Layer displacement',
1989	'Displace([XY])' : ('Debye-Scherrer sample displacement \\1',0.1),
1990	'Lam' : ('Wavelength',1e-6),
1991	'I\\(L2\\)\\/I\\(L1\\)' : ('Ka2/Ka1 intensity ratio',0.001),
1992	'Polariz\\.' : ('Polarization correction',1e-3),
1993	'SH/L' : ('FCJ peak asymmetry correction',1e-4),
1994	'([UVW])$' : ('Gaussian instrument broadening \\1',1e-5),
1995	'([XYZ])$' : ('Cauchy instrument broadening \\1',1e-5),
1996	'Zero' : 'Debye-Scherrer zero correction',
1997	'Shift' : 'Bragg-Brentano sample displ.',
1998	'SurfRoughA' : 'Bragg-Brenano surface roughness A',
1999	'SurfRoughB' : 'Bragg-Brenano surface roughness B',
2000	'Transparency' : 'Bragg-Brentano sample tranparency',
2001	'DebyeA' : 'Debye model amplitude',
2002	'DebyeR' : 'Debye model radius',
2003	'DebyeU' : 'Debye model Uiso',
2004	'RBV.*' : 'Vector rigid body parameter',
2005	'RBVO([aijk])' : 'Vector rigid body orientation parameter \\1',
2006	'RBVP([xyz])' : 'Vector rigid body \\1 position parameter',
2007	'RBVf' : 'Vector rigid body site fraction',
2008	'RBV([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
2009	'RBVU' : 'Residue rigid body group Uiso param.',
2010	'RBRO([aijk])' : 'Residue rigid body orientation parameter \\1',
2011	'RBRP([xyz])' : 'Residue rigid body \\1 position parameter',
2012	'RBRTr;.*' : 'Residue rigid body torsion parameter',
2013	'RBRf' : 'Residue rigid body site fraction',
2014	'RBR([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
2015	'RBRU' : 'Residue rigid body group Uiso param.',
2016	'constr([0-9]*)' : 'Generated degree of freedom from constraint',
2017	'nv-(.+)' : 'New variable assignment with name \\1',
2018	# supersymmetry parameters p::<var>:a:o 'Flen','Fcent'?
2019	'mV([0-2])$' : 'Modulation vector component \\1',
2020	'Fsin' : 'Sin site fraction modulation',
2021	'Fcos' : 'Cos site fraction modulation',
2022	'Fzero' : 'Crenel function offset', #may go away
2023	'Fwid' : 'Crenel function width',
2024	'Tmin' : 'ZigZag/Block min location',
2025	'Tmax' : 'ZigZag/Block max location',
2026	'([XYZ])max': 'ZigZag/Block max value for \\1',
2027	'([XYZ])sin' : 'Sin position wave for \\1',
2028	'([XYZ])cos' : 'Cos position wave for \\1',
2029	'U([123][123])sin$' : 'Sin thermal wave for U\\1',
2030	'U([123][123])cos$' : 'Cos thermal wave for U\\1',
2031	'M([XYZ])sin$' : 'Sin mag. moment wave for \\1',
2032	'M([XYZ])cos$' : 'Cos mag. moment wave for \\1',
2033	# PDF peak parms (l:<var>;l = peak no.)
2034	'PDFpos' : 'PDF peak position',
2035	'PDFmag' : 'PDF peak magnitude',
2036	'PDFsig' : 'PDF peak std. dev.',
2037	# SASD vars (l:<var>;l = component)
2038	'Aspect ratio' : 'Particle aspect ratio',
2039	'Length' : 'Cylinder length',
2040	'Diameter' : 'Cylinder/disk diameter',
2041	'Thickness' : 'Disk thickness',
2042	'Shell thickness' : 'Multiplier to get inner(<1) or outer(>1) sphere radius',
2043	'Dist' : 'Interparticle distance',
2044	'VolFr' : 'Dense scatterer volume fraction',
2045	'epis' : 'Sticky sphere epsilon',
2046	'Sticky' : 'Stickyness',
2047	'Depth' : 'Well depth',
2048	'Width' : 'Well width',
2049	'Volume' : 'Particle volume',
2050	'Radius' : 'Sphere/cylinder/disk radius',
2051	'Mean' : 'Particle mean radius',
2052	'StdDev' : 'Standard deviation in Mean',
2053	'G$': 'Guinier prefactor',
2054	'Rg$': 'Guinier radius of gyration',
2055	'B$': 'Porod prefactor',
2056	'P$': 'Porod power',
2057	'Cutoff': 'Porod cutoff',
2058	'PkInt': 'Bragg peak intensity',
2059	'PkPos': 'Bragg peak position',
2060	'PkSig': 'Bragg peak sigma',
2061	'PkGam': 'Bragg peak gamma',
2062	'e([12][12])' : 'strain tensor e\\1', # strain vars e11, e22, e12
2063	'Dcalc': 'Calc. d-spacing',
2064	'Back$': 'background parameter',
2065	'pos$': 'peak position',
2066	'int$': 'peak intensity',
2067	'WgtFrac':'phase weight fraction',
2068	'alpha':'TOF profile term',
2069	'alpha-[01]':'Pink profile term',
2070	'beta-[01q]':'TOF/Pink profile term',
2071	'sig-[012q]':'TOF profile term',
2072	'dif[ABC]':'TOF to d-space calibration',
2073	'C\\([0-9],[0-9]\\)' : 'spherical harmonics preferred orientation coef.',
2074	'Pressure': 'Pressure level for measurement in MPa',
2075	'Temperature': 'T value for measurement, K',
2076	'FreePrm([123])': 'User defined measurement parameter \\1',
2077	'Gonio. radius': 'Distance from sample to detector, mm',
2078	}.items():
2079	# Needs documentation: HAP: LeBail, newLeBail
2080	# hist: Azimuth, Chi, Omega, Phi, Bank, nDebye, nPeaks
2081
2082	if len(value) == 2:
2083	#VarDesc[key] = value[0]
2084	reVarDesc[re.compile(key)] = value[0]
2085	reVarStep[re.compile(key)] = value[1]
2086	else:
2087	#VarDesc[key] = value
2088	reVarDesc[re.compile(key)] = value
2089
2090	def removeNonRefined(parmList):
2091	'''Remove items from variable list that are not refined and should not
2092	appear as options for constraints
2093
2094	:param list parmList: a list of strings of form "p:h:VAR:a" where
2095	VAR is the variable name
2096
2097	:returns: a list after removing variables where VAR matches a
2098	entry in local variable NonRefinedList
2099	'''
2100	NonRefinedList = ['Omega','Type','Chi','Phi', 'Azimuth','Gonio. radius',
2101	'Lam1','Lam2','Back','Temperature','Pressure',
2102	'FreePrm1','FreePrm2','FreePrm3',
2103	'Source','nPeaks','LeBail','newLeBail','Bank',
2104	'nDebye', #'',
2105	]
2106	return [prm for prm in parmList if prm.split(':')[2] not in NonRefinedList]
2107
2108	def getDescr(name):
2109	'''Return a short description for a GSAS-II variable
2110
2111	:param str name: The descriptive part of the variable name without colons (:)
2112
2113	:returns: a short description or None if not found
2114	'''
2115
2116	CompileVarDesc() # compile the regular expressions, if needed
2117	for key in reVarDesc:
2118	m = key.match(name)
2119	if m:
2120	reVarDesc[key]
2121	return m.expand(reVarDesc[key])
2122	return None
2123
2124	def getVarStep(name,parmDict=None):
2125	'''Return a step size for computing the derivative of a GSAS-II variable
2126
2127	:param str name: A complete variable name (with colons, :)
2128	:param dict parmDict: A dict with parameter values or None (default)
2129
2130	:returns: a float that should be an appropriate step size, either from
2131	the value supplied in :func:`CompileVarDesc` or based on the value for
2132	name in parmDict, if supplied. If not found or the value is zero,
2133	a default value of 1e-5 is used. If parmDict is None (default) and
2134	no value is provided in :func:`CompileVarDesc`, then None is returned.
2135	'''
2136	CompileVarDesc() # compile the regular expressions, if needed
2137	for key in reVarStep:
2138	m = key.match(name)
2139	if m:
2140	return reVarStep[key]
2141	if parmDict is None: return None
2142	val = parmDict.get(key,0.0)
2143	if abs(val) > 0.05:
2144	return abs(val)/1000.
2145	else:
2146	return 1e-5
2147
2148	def GenWildCard(varlist):
2149	'''Generate wildcard versions of G2 variables. These introduce '*'
2150	for a phase, histogram or atom number (but only for one of these
2151	fields) but only when there is more than one matching variable in the
2152	input variable list. So if the input is this::
2153
2154	varlist = ['0::AUiso:0', '0::AUiso:1', '1::AUiso:0']
2155
2156	then the output will be this::
2157
2158	wildList = ['::AUiso:0', '0::AUiso:']
2159
2160	:param list varlist: an input list of GSAS-II variable names
2161	(such as 0::AUiso:0)
2162
2163	:returns: wildList, the generated list of wild card variable names.
2164	'''
2165	wild = []
2166	for i in (0,1,3):
2167	currentL = varlist[:]
2168	while currentL:
2169	item1 = currentL.pop(0)
2170	i1splt = item1.split(':')
2171	if i >= len(i1splt): continue
2172	if i1splt[i]:
2173	nextL = []
2174	i1splt[i] = '[0-9]+'
2175	rexp = re.compile(':'.join(i1splt))
2176	matchlist = [item1]
2177	for nxtitem in currentL:
2178	if rexp.match(nxtitem):
2179	matchlist += [nxtitem]
2180	else:
2181	nextL.append(nxtitem)
2182	if len(matchlist) > 1:
2183	i1splt[i] = '*'
2184	wild.append(':'.join(i1splt))
2185	currentL = nextL
2186	return wild
2187
2188	def LookupWildCard(varname,varlist):
2189	'''returns a list of variable names from list varname
2190	that match wildcard name in varname
2191
2192	:param str varname: a G2 variable name containing a wildcard
2193	(such as \\*::var)
2194	:param list varlist: the list of all variable names used in
2195	the current project
2196	:returns: a list of matching GSAS-II variables (may be empty)
2197	'''
2198	rexp = re.compile(varname.replace('*','[0-9]+'))
2199	return sorted([var for var in varlist if rexp.match(var)])
2200
2201	def prmLookup(name,prmDict):
2202	'''Looks for a parameter in a min/max dictionary, optionally
2203	considering a wild card for histogram or atom number (use of
2204	both will never occur at the same time).
2205
2206	:param name: a GSAS-II parameter name (str, see :func:`getVarDescr`
2207	and :func:`CompileVarDesc`) or a :class:`G2VarObj` object.
2208	:param dict prmDict: a min/max dictionary, (parmMinDict
2209	or parmMaxDict in Controls) where keys are :class:`G2VarObj`
2210	objects.
2211	:returns: Two values, (matchname, value), are returned where:
2212
2213	* matchname (str) is the :class:`G2VarObj` object
2214	corresponding to the actual matched name,
2215	which could contain a wildcard even if name does not; and
2216	* value (float) which contains the parameter limit.
2217	'''
2218	for key,value in prmDict.items():
2219	if str(key) == str(name): return key,value
2220	if key == name: return key,value
2221	return None,None
2222
2223
2224	def _lookup(dic,key):
2225	'''Lookup a key in a dictionary, where None returns an empty string
2226	but an unmatched key returns a question mark. Used in :class:`G2VarObj`
2227	'''
2228	if key is None:
2229	return ""
2230	elif key == "*":
2231	return "*"
2232	else:
2233	return dic.get(key,'?')
2234
2235	def SortVariables(varlist):
2236	'''Sorts variable names in a sensible manner
2237	'''
2238	def cvnnums(var):
2239	v = []
2240	for i in var.split(':'):
2241	# if i == '' or i == '*':
2242	# v.append(-1)
2243	# continue
2244	try:
2245	v.append(int(i))
2246	except:
2247	v.append(-1)
2248	return v
2249	return sorted(varlist,key=cvnnums)
2250
2251	class G2VarObj(object):
2252	'''Defines a GSAS-II variable either using the phase/atom/histogram
2253	unique Id numbers or using a character string that specifies
2254	variables by phase/atom/histogram number (which can change).
2255	Note that :func:`GSASIIstrIO.GetUsedHistogramsAndPhases`,
2256	which calls :func:`IndexAllIds` (or
2257	:func:`GSASIIscriptable.G2Project.index_ids`) should be used to
2258	(re)load the current Ids
2259	before creating or later using the G2VarObj object.
2260
2261	This can store rigid body variables, but does not translate the residue # and
2262	body # to/from random Ids
2263
2264	A :class:`G2VarObj` object can be created with a single parameter:
2265
2266	:param str/tuple varname: a single value can be used to create a :class:`G2VarObj`
2267	object. If a string, it must be of form "p:h:var" or "p:h:var:a", where
2268
2269	* p is the phase number (which may be left blank or may be '*' to indicate all phases);
2270	* h is the histogram number (which may be left blank or may be '*' to indicate all histograms);
2271	* a is the atom number (which may be left blank in which case the third colon is omitted).
2272	The atom number can be specified as '' if a phase number is specified (not as '').
2273	For rigid body variables, specify a will be a string of form "residue:body#"
2274
2275	Alternately a single tuple of form (Phase,Histogram,VarName,AtomID) can be used, where
2276	Phase, Histogram, and AtomID are None or are ranId values (or one can be '*')
2277	and VarName is a string. Note that if Phase is '*' then the AtomID is an atom number.
2278	For a rigid body variables, AtomID is a string of form "residue:body#".
2279
2280	If four positional arguments are supplied, they are:
2281
2282	:param str/int phasenum: The number for the phase (or None or '*')
2283	:param str/int histnum: The number for the histogram (or None or '*')
2284	:param str varname: a single value can be used to create a :class:`G2VarObj`
2285	:param str/int atomnum: The number for the atom (or None or '*')
2286
2287	'''
2288	IDdict = {}
2289	IDdict['phases'] = {}
2290	IDdict['hists'] = {}
2291	IDdict['atoms'] = {}
2292	def __init__(self,*args):
2293	self.phase = None
2294	self.histogram = None
2295	self.name = ''
2296	self.atom = None
2297	if len(args) == 1 and (type(args[0]) is list or type(args[0]) is tuple) and len(args[0]) == 4:
2298	# single arg with 4 values
2299	self.phase,self.histogram,self.name,self.atom = args[0]
2300	elif len(args) == 1 and ':' in args[0]:
2301	#parse a string
2302	lst = args[0].split(':')
2303	if lst[0] == '*':
2304	self.phase = '*'
2305	if len(lst) > 3:
2306	self.atom = lst[3]
2307	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
2308	elif lst[1] == '*':
2309	self.histogram = '*'
2310	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
2311	else:
2312	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
2313	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
2314	if len(lst) == 4:
2315	if lst[3] == '*':
2316	self.atom = '*'
2317	else:
2318	self.atom = AtomIdLookup[lst[0]].get(lst[3],[None,None])[1]
2319	elif len(lst) == 5:
2320	self.atom = lst[3]+":"+lst[4]
2321	elif len(lst) == 3:
2322	pass
2323	else:
2324	raise Exception("Incorrect number of colons in var name "+str(args[0]))
2325	self.name = lst[2]
2326	elif len(args) == 4:
2327	if args[0] == '*':
2328	self.phase = '*'
2329	self.atom = args[3]
2330	else:
2331	self.phase = PhaseIdLookup.get(str(args[0]),[None,None])[1]
2332	if args[3] == '*':
2333	self.atom = '*'
2334	elif args[0] is not None:
2335	self.atom = AtomIdLookup[args[0]].get(str(args[3]),[None,None])[1]
2336	if args[1] == '*':
2337	self.histogram = '*'
2338	else:
2339	self.histogram = HistIdLookup.get(str(args[1]),[None,None])[1]
2340	self.name = args[2]
2341	else:
2342	raise Exception("Incorrectly called GSAS-II parameter name")
2343
2344	#print "DEBUG: created ",self.phase,self.histogram,self.name,self.atom
2345
2346	def __str__(self):
2347	return self.varname()
2348
2349	def __hash__(self):
2350	'Allow G2VarObj to be a dict key by implementing hashing'
2351	return hash(self.varname())
2352
2353	def varname(self,hist=None):
2354	'''Formats the GSAS-II variable name as a "traditional" GSAS-II variable
2355	string (p:h:<var>:a) or (p:h:<var>)
2356
2357	:param str/int hist: if specified, overrides the histogram number
2358	with the specified value
2359	:returns: the variable name as a str
2360	'''
2361	a = ""
2362	if self.phase == "*":
2363	ph = "*"
2364	if self.atom:
2365	a = ":" + str(self.atom)
2366	else:
2367	ph = _lookup(PhaseRanIdLookup,self.phase)
2368	if self.atom == '*':
2369	a = ':*'
2370	elif self.atom:
2371	if ":" in str(self.atom):
2372	a = ":" + str(self.atom)
2373	elif ph in AtomRanIdLookup:
2374	a = ":" + AtomRanIdLookup[ph].get(self.atom,'?')
2375	else:
2376	a = ":?"
2377	if hist is not None and self.histogram:
2378	hist = str(hist)
2379	elif self.histogram == "*":
2380	hist = "*"
2381	else:
2382	hist = _lookup(HistRanIdLookup,self.histogram)
2383	s = (ph + ":" + hist + ":" + str(self.name)) + a
2384	return s
2385
2386	def __repr__(self):
2387	'''Return the detailed contents of the object
2388	'''
2389	s = "<"
2390	if self.phase == '*':
2391	s += "Phases: all; "
2392	if self.atom is not None:
2393	if ":" in str(self.atom):
2394	s += "Rigid body" + str(self.atom) + "; "
2395	else:
2396	s += "Atom #" + str(self.atom) + "; "
2397	elif self.phase is not None:
2398	ph = _lookup(PhaseRanIdLookup,self.phase)
2399	s += "Phase: rId=" + str(self.phase) + " (#"+ ph + "); "
2400	if self.atom == '*':
2401	s += "Atoms: all; "
2402	elif ":" in str(self.atom):
2403	s += "Rigid body" + str(self.atom) + "; "
2404	elif self.atom is not None:
2405	s += "Atom rId=" + str(self.atom)
2406	if ph in AtomRanIdLookup:
2407	s += " (#" + AtomRanIdLookup[ph].get(self.atom,'?') + "); "
2408	else:
2409	s += " (#? -- not found!); "
2410	if self.histogram == '*':
2411	s += "Histograms: all; "
2412	elif self.histogram is not None:
2413	hist = _lookup(HistRanIdLookup,self.histogram)
2414	s += "Histogram: rId=" + str(self.histogram) + " (#"+ hist + "); "
2415	s += 'Variable name="' + str(self.name) + '">'
2416	return s+" ("+self.varname()+")"
2417
2418	def __eq__(self, other):
2419	'''Allow comparison of G2VarObj to other G2VarObj objects or strings.
2420	If any field is a wildcard ('*') that field matches.
2421	'''
2422	if type(other) is str:
2423	other = G2VarObj(other)
2424	elif type(other) is not G2VarObj:
2425	raise Exception("Invalid type ({}) for G2VarObj comparison with {}"
2426	.format(type(other),other))
2427	if self.phase != other.phase and self.phase != '' and other.phase != '':
2428	return False
2429	if self.histogram != other.histogram and self.histogram != '' and other.histogram != '':
2430	return False
2431	if self.atom != other.atom and self.atom != '' and other.atom != '':
2432	return False
2433	if self.name != other.name:
2434	return False
2435	return True
2436
2437	def fmtVarByMode(self, seqmode, note, warnmsg):
2438	'''Format a parameter object for display. Note that these changes
2439	are only temporary and are only shown only when the Constraints
2440	data tree is selected.
2441
2442	* In a non-sequential refinement or where the mode is 'use-all', the
2443	name is converted unchanged to a str
2444	* In a sequential refinement when the mode is 'wildcards-only' the
2445	name is converted unchanged to a str but a warning is added
2446	for non-wildcarded HAP or Histogram parameters
2447	* In a sequential refinement or where the mode is 'auto-wildcard',
2448	a histogram number is converted to a wildcard (*) and then
2449	converted to str
2450
2451	:param str mode: the sequential mode (see above)
2452	:param str note: value displayed on the line of the constraint/equiv.
2453	:param str warnmsg: a message saying the constraint is not used
2454
2455	:returns: varname, explain, note, warnmsg (all str values) where:
2456
2457	* varname is the parameter expressed as a string,
2458	* explain is blank unless there is a warning explanation about
2459	the parameter or blank
2460	* note is the previous value unless overridden
2461	* warnmsg is the previous value unless overridden
2462	'''
2463	explain = ''
2464	s = self.varname()
2465	if seqmode == 'auto-wildcard':
2466	if self.histogram: s = self.varname('*')
2467	elif seqmode == 'wildcards-only' and self.histogram:
2468	if self.histogram != '*':
2469	warnmsg = 'Ignored due to use of a non-wildcarded histogram number'
2470	note = 'Ignored'
2471	explain = '\nIgnoring: '+self.varname()+' does not contain a wildcard.\n'
2472	elif seqmode != 'use-all' and seqmode != 'wildcards-only':
2473	print('Unexpected mode',seqmode,' in fmtVarByMode')
2474	return s,explain,note,warnmsg
2475
2476	def _show(self):
2477	'For testing, shows the current lookup table'
2478	print ('phases'+ self.IDdict['phases'])
2479	print ('hists'+ self.IDdict['hists'])
2480	print ('atomDict'+ self.IDdict['atoms'])
2481
2482	#==========================================================================
2483	def SetDefaultSample():
2484	'Fills in default items for the Sample dictionary for Debye-Scherrer & SASD'
2485	return {
2486	'InstrName':'',
2487	'ranId':ran.randint(0,sys.maxsize),
2488	'Scale':[1.0,True],'Type':'Debye-Scherrer','Absorption':[0.0,False],
2489	'DisplaceX':[0.0,False],'DisplaceY':[0.0,False],
2490	'Temperature':300.,'Pressure':0.1,'Time':0.0,
2491	'FreePrm1':0.,'FreePrm2':0.,'FreePrm3':0.,
2492	'Gonio. radius':200.0,
2493	'Omega':0.0,'Chi':0.0,'Phi':0.0,'Azimuth':0.0,
2494	#SASD items
2495	'Materials':[{'Name':'vacuum','VolFrac':1.0,},{'Name':'vacuum','VolFrac':0.0,}],
2496	'Thick':1.0,'Contrast':[0.0,0.0], #contrast & anomalous contrast
2497	'Trans':1.0, #measured transmission
2498	'SlitLen':0.0, #Slit length - in Q(A-1)
2499	}
2500	######################################################################
2501	class ImportBaseclass(object):
2502	'''Defines a base class for the reading of input files (diffraction
2503	data, coordinates,...). See :ref:`Writing a Import Routine<import_routines>`
2504	for an explanation on how to use a subclass of this class.
2505	'''
2506	class ImportException(Exception):
2507	'''Defines an Exception that is used when an import routine hits an expected error,
2508	usually in .Reader.
2509
2510	Good practice is that the Reader should define a value in self.errors that
2511	tells the user some information about what is wrong with their file.
2512	'''
2513	pass
2514
2515	UseReader = True # in __init__ set value of self.UseReader to False to skip use of current importer
2516	def __init__(self,formatName,longFormatName=None,
2517	extensionlist=[],strictExtension=False,):
2518	self.formatName = formatName # short string naming file type
2519	if longFormatName: # longer string naming file type
2520	self.longFormatName = longFormatName
2521	else:
2522	self.longFormatName = formatName
2523	# define extensions that are allowed for the file type
2524	# for windows, remove any extensions that are duplicate, as case is ignored
2525	if sys.platform == 'windows' and extensionlist:
2526	extensionlist = list(set([s.lower() for s in extensionlist]))
2527	self.extensionlist = extensionlist
2528	# If strictExtension is True, the file will not be read, unless
2529	# the extension matches one in the extensionlist
2530	self.strictExtension = strictExtension
2531	self.errors = ''
2532	self.warnings = ''
2533	self.SciPy = False #image reader needed scipy
2534	# used for readers that will use multiple passes to read
2535	# more than one data block
2536	self.repeat = False
2537	self.selections = []
2538	self.repeatcount = 0
2539	self.readfilename = '?'
2540	self.scriptable = False
2541	#print 'created',self.__class__
2542
2543	def ReInitialize(self):
2544	'Reinitialize the Reader to initial settings'
2545	self.errors = ''
2546	self.warnings = ''
2547	self.SciPy = False #image reader needed scipy
2548	self.repeat = False
2549	self.repeatcount = 0
2550	self.readfilename = '?'
2551
2552
2553	# def Reader(self, filename, filepointer, ParentFrame=None, **unused):
2554	# '''This method must be supplied in the child class to read the file.
2555	# if the read fails either return False or raise an Exception
2556	# preferably of type ImportException.
2557	# '''
2558	# #start reading
2559	# raise ImportException("Error occurred while...")
2560	# self.errors += "Hint for user on why the error occur
2561	# return False # if an error occurs
2562	# return True # if read OK
2563
2564	def ExtensionValidator(self, filename):
2565	'''This methods checks if the file has the correct extension
2566
2567	:returns:
2568
2569	* False if this filename will not be supported by this reader (only
2570	when strictExtension is True)
2571	* True if the extension matches the list supplied by the reader
2572	* None if the reader allows un-registered extensions
2573
2574	'''
2575	if filename:
2576	ext = os.path.splitext(filename)[1]
2577	if not ext and self.strictExtension: return False
2578	for ext in self.extensionlist:
2579	if sys.platform == 'windows':
2580	if filename.lower().endswith(ext): return True
2581	else:
2582	if filename.endswith(ext): return True
2583	if self.strictExtension:
2584	return False
2585	else:
2586	return None
2587
2588	def ContentsValidator(self, filename):
2589	'''This routine will attempt to determine if the file can be read
2590	with the current format.
2591	This will typically be overridden with a method that
2592	takes a quick scan of [some of]
2593	the file contents to do a "sanity" check if the file
2594	appears to match the selected format.
2595	the file must be opened here with the correct format (binary/text)
2596	'''
2597	#filepointer.seek(0) # rewind the file pointer
2598	return True
2599
2600	def CIFValidator(self, filepointer):
2601	'''A :meth:`ContentsValidator` for use to validate CIF files.
2602	'''
2603	filepointer.seek(0)
2604	for i,l in enumerate(filepointer):
2605	if i >= 1000: return True
2606	'''Encountered only blank lines or comments in first 1000
2607	lines. This is unlikely, but assume it is CIF anyway, since we are
2608	even less likely to find a file with nothing but hashes and
2609	blank lines'''
2610	line = l.strip()
2611	if len(line) == 0: # ignore blank lines
2612	continue
2613	elif line.startswith('#'): # ignore comments
2614	continue
2615	elif line.startswith('data_'): # on the right track, accept this file
2616	return True
2617	else: # found something invalid
2618	self.errors = 'line '+str(i+1)+' contains unexpected data:\n'
2619	if all([ord(c) < 128 and ord(c) != 0 for c in str(l)]): # show only if ASCII
2620	self.errors += ' '+str(l)
2621	else:
2622	self.errors += ' (binary)'
2623	self.errors += '\n Note: a CIF should only have blank lines or comments before'
2624	self.errors += '\n a data_ statement begins a block.'
2625	return False
2626
2627	######################################################################
2628	class ImportPhase(ImportBaseclass):
2629	'''Defines a base class for the reading of files with coordinates
2630
2631	Objects constructed that subclass this (in import/G2phase_*.py etc.) will be used
2632	in :meth:`GSASIIdataGUI.GSASII.OnImportPhase` and in
2633	:func:`GSASIIscriptable.import_generic`.
2634	See :ref:`Writing a Import Routine<import_routines>`
2635	for an explanation on how to use this class.
2636
2637	'''
2638	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2639	strictExtension=False,):
2640	# call parent __init__
2641	ImportBaseclass.__init__(self,formatName,longFormatName,
2642	extensionlist,strictExtension)
2643	self.Phase = None # a phase must be created with G2IO.SetNewPhase in the Reader
2644	self.SymOps = {} # specified when symmetry ops are in file (e.g. CIF)
2645	self.Constraints = None
2646
2647	######################################################################
2648	class ImportStructFactor(ImportBaseclass):
2649	'''Defines a base class for the reading of files with tables
2650	of structure factors.
2651
2652	Structure factors are read with a call to :meth:`GSASIIdataGUI.GSASII.OnImportSfact`
2653	which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`, which calls
2654	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2655	:meth:`Reader`.
2656
2657	See :ref:`Writing a Import Routine<import_routines>`
2658	for an explanation on how to use import classes in general. The specifics
2659	for reading a structure factor histogram require that
2660	the ``Reader()`` routine in the import
2661	class need to do only a few things: It
2662	should load :attr:`RefDict` item ``'RefList'`` with the reflection list,
2663	and set :attr:`Parameters` with the instrument parameters
2664	(initialized with :meth:`InitParameters` and set with :meth:`UpdateParameters`).
2665	'''
2666	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2667	strictExtension=False,):
2668	ImportBaseclass.__init__(self,formatName,longFormatName,
2669	extensionlist,strictExtension)
2670
2671	# define contents of Structure Factor entry
2672	self.Parameters = []
2673	'self.Parameters is a list with two dicts for data parameter settings'
2674	self.InitParameters()
2675	self.RefDict = {'RefList':[],'FF':{},'Super':0}
2676	self.Banks = [] #for multi bank data (usually TOF)
2677	'''self.RefDict is a dict containing the reflection information, as read from the file.
2678	Item 'RefList' contains the reflection information. See the
2679	:ref:`Single Crystal Reflection Data Structure<XtalRefl_table>`
2680	for the contents of each row. Dict element 'FF'
2681	contains the form factor values for each element type; if this entry
2682	is left as initialized (an empty list) it will be initialized as needed later.
2683	'''
2684	def ReInitialize(self):
2685	'Reinitialize the Reader to initial settings'
2686	ImportBaseclass.ReInitialize(self)
2687	self.InitParameters()
2688	self.Banks = [] #for multi bank data (usually TOF)
2689	self.RefDict = {'RefList':[],'FF':{},'Super':0}
2690
2691	def InitParameters(self):
2692	'initialize the instrument parameters structure'
2693	Lambda = 0.70926
2694	HistType = 'SXC'
2695	self.Parameters = [{'Type':[HistType,HistType], # create the structure
2696	'Lam':[Lambda,Lambda]
2697	}, {}]
2698	'Parameters is a list with two dicts for data parameter settings'
2699
2700	def UpdateParameters(self,Type=None,Wave=None):
2701	'Revise the instrument parameters'
2702	if Type is not None:
2703	self.Parameters[0]['Type'] = [Type,Type]
2704	if Wave is not None:
2705	self.Parameters[0]['Lam'] = [Wave,Wave]
2706
2707	######################################################################
2708	class ImportPowderData(ImportBaseclass):
2709	'''Defines a base class for the reading of files with powder data.
2710
2711	Objects constructed that subclass this (in import/G2pwd_*.py etc.) will be used
2712	in :meth:`GSASIIdataGUI.GSASII.OnImportPowder` and in
2713	:func:`GSASIIscriptable.import_generic`.
2714	See :ref:`Writing a Import Routine<import_routines>`
2715	for an explanation on how to use this class.
2716	'''
2717	def __init__(self,formatName,longFormatName=None,
2718	extensionlist=[],strictExtension=False,):
2719	ImportBaseclass.__init__(self,formatName,longFormatName,
2720	extensionlist,strictExtension)
2721	self.clockWd = None # used in TOF
2722	self.ReInitialize()
2723
2724	def ReInitialize(self):
2725	'Reinitialize the Reader to initial settings'
2726	ImportBaseclass.ReInitialize(self)
2727	self.powderentry = ['',None,None] # (filename,Pos,Bank)
2728	self.powderdata = [] # Powder dataset
2729	'''A powder data set is a list with items [x,y,w,yc,yb,yd]:
2730	np.array(x), # x-axis values
2731	np.array(y), # powder pattern intensities
2732	np.array(w), # 1/sig(intensity)^2 values (weights)
2733	np.array(yc), # calc. intensities (zero)
2734	np.array(yb), # calc. background (zero)
2735	np.array(yd), # obs-calc profiles
2736	'''
2737	self.comments = []
2738	self.idstring = ''
2739	self.Sample = SetDefaultSample() # default sample parameters
2740	self.Controls = {} # items to be placed in top-level Controls
2741	self.GSAS = None # used in TOF
2742	self.repeat_instparm = True # Should a parm file be
2743	# used for multiple histograms?
2744	self.instparm = None # name hint from file of instparm to use
2745	self.instfile = '' # full path name to instrument parameter file
2746	self.instbank = '' # inst parm bank number
2747	self.instmsg = '' # a label that gets printed to show
2748	# where instrument parameters are from
2749	self.numbanks = 1
2750	self.instdict = {} # place items here that will be transferred to the instrument parameters
2751	self.pwdparms = {} # place parameters that are transferred directly to the tree
2752	# here (typically from an existing GPX file)
2753	######################################################################
2754	class ImportSmallAngleData(ImportBaseclass):
2755	'''Defines a base class for the reading of files with small angle data.
2756	See :ref:`Writing a Import Routine<import_routines>`
2757	for an explanation on how to use this class.
2758	'''
2759	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2760	strictExtension=False,):
2761
2762	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2763	strictExtension)
2764	self.ReInitialize()
2765
2766	def ReInitialize(self):
2767	'Reinitialize the Reader to initial settings'
2768	ImportBaseclass.ReInitialize(self)
2769	self.smallangleentry = ['',None,None] # (filename,Pos,Bank)
2770	self.smallangledata = [] # SASD dataset
2771	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2772	np.array(x), # x-axis values
2773	np.array(y), # powder pattern intensities
2774	np.array(w), # 1/sig(intensity)^2 values (weights)
2775	np.array(yc), # calc. intensities (zero)
2776	np.array(yd), # obs-calc profiles
2777	np.array(yb), # preset bkg
2778	'''
2779	self.comments = []
2780	self.idstring = ''
2781	self.Sample = SetDefaultSample()
2782	self.GSAS = None # used in TOF
2783	self.clockWd = None # used in TOF
2784	self.numbanks = 1
2785	self.instdict = {} # place items here that will be transferred to the instrument parameters
2786
2787	######################################################################
2788	class ImportReflectometryData(ImportBaseclass):
2789	'''Defines a base class for the reading of files with reflectometry data.
2790	See :ref:`Writing a Import Routine<import_routines>`
2791	for an explanation on how to use this class.
2792	'''
2793	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2794	strictExtension=False,):
2795
2796	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2797	strictExtension)
2798	self.ReInitialize()
2799
2800	def ReInitialize(self):
2801	'Reinitialize the Reader to initial settings'
2802	ImportBaseclass.ReInitialize(self)
2803	self.reflectometryentry = ['',None,None] # (filename,Pos,Bank)
2804	self.reflectometrydata = [] # SASD dataset
2805	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2806	np.array(x), # x-axis values
2807	np.array(y), # powder pattern intensities
2808	np.array(w), # 1/sig(intensity)^2 values (weights)
2809	np.array(yc), # calc. intensities (zero)
2810	np.array(yd), # obs-calc profiles
2811	np.array(yb), # preset bkg
2812	'''
2813	self.comments = []
2814	self.idstring = ''
2815	self.Sample = SetDefaultSample()
2816	self.GSAS = None # used in TOF
2817	self.clockWd = None # used in TOF
2818	self.numbanks = 1
2819	self.instdict = {} # place items here that will be transferred to the instrument parameters
2820
2821	######################################################################
2822	class ImportPDFData(ImportBaseclass):
2823	'''Defines a base class for the reading of files with PDF G(R) data.
2824	See :ref:`Writing a Import Routine<import_routines>`
2825	for an explanation on how to use this class.
2826	'''
2827	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2828	strictExtension=False,):
2829
2830	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2831	strictExtension)
2832	self.ReInitialize()
2833
2834	def ReInitialize(self):
2835	'Reinitialize the Reader to initial settings'
2836	ImportBaseclass.ReInitialize(self)
2837	self.pdfentry = ['',None,None] # (filename,Pos,Bank)
2838	self.pdfdata = [] # PDF G(R) dataset
2839	'''A pdf g(r) data set is a list with items [x,y]:
2840	np.array(x), # r-axis values
2841	np.array(y), # pdf g(r)
2842	'''
2843	self.comments = []
2844	self.idstring = ''
2845	self.numbanks = 1
2846
2847	######################################################################
2848	class ImportImage(ImportBaseclass):
2849	'''Defines a base class for the reading of images
2850
2851	Images are read in only these places:
2852
2853	* Initial reading is typically done from a menu item
2854	with a call to :meth:`GSASIIdataGUI.GSASII.OnImportImage`
2855	which in turn calls :meth:`GSASIIdataGUI.GSASII.OnImportGeneric`. That calls
2856	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2857	:meth:`Reader`. This returns a list of reader objects for each read image.
2858	Also used in :func:`GSASIIscriptable.import_generic`.
2859
2860	* Images are read alternatively in :func:`GSASIIIO.ReadImages`, which puts image info
2861	directly into the data tree.
2862
2863	* Images are reloaded with :func:`GSASIIIO.GetImageData`.
2864
2865	When reading an image, the ``Reader()`` routine in the ImportImage class
2866	should set:
2867
2868	* :attr:`Comments`: a list of strings (str),
2869	* :attr:`Npix`: the number of pixels in the image (int),
2870	* :attr:`Image`: the actual image as a numpy array (np.array)
2871	* :attr:`Data`: a dict defining image parameters (dict). Within this dict the following
2872	data items are needed:
2873
2874	* 'pixelSize': size of each pixel in microns (such as ``[200.,200.]``.
2875	* 'wavelength': wavelength in :math:`\\AA`.
2876	* 'distance': distance of detector from sample in cm.
2877	* 'center': uncalibrated center of beam on detector (such as ``[204.8,204.8]``.
2878	* 'size': size of image (such as ``[2048,2048]``).
2879	* 'ImageTag': image number or other keyword used to retrieve image from
2880	a multi-image data file (defaults to ``1`` if not specified).
2881	* 'sumfile': holds sum image file name if a sum was produced from a multi image file
2882
2883	optional data items:
2884
2885	* :attr:`repeat`: set to True if there are additional images to
2886	read in the file, False otherwise
2887	* :attr:`repeatcount`: set to the number of the image.
2888
2889	Note that the above is initialized with :meth:`InitParameters`.
2890	(Also see :ref:`Writing a Import Routine<import_routines>`
2891	for an explanation on how to use import classes in general.)
2892	'''
2893	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2894	strictExtension=False,):
2895	ImportBaseclass.__init__(self,formatName,longFormatName,
2896	extensionlist,strictExtension)
2897	self.InitParameters()
2898
2899	def ReInitialize(self):
2900	'Reinitialize the Reader to initial settings -- not used at present'
2901	ImportBaseclass.ReInitialize(self)
2902	self.InitParameters()
2903
2904	def InitParameters(self):
2905	'initialize the instrument parameters structure'
2906	self.Comments = ['No comments']
2907	self.Data = {'samplechangerpos':0.0,'det2theta':0.0,'Gain map':''}
2908	self.Npix = 0
2909	self.Image = None
2910	self.repeat = False
2911	self.repeatcount = 1
2912	self.sumfile = ''
2913
2914	def LoadImage(self,ParentFrame,imagefile,imagetag=None):
2915	'''Optionally, call this after reading in an image to load it into the tree.
2916	This saves time by preventing a reread of the same information.
2917	'''
2918	if ParentFrame:
2919	ParentFrame.ImageZ = self.Image # store the image for plotting
2920	ParentFrame.oldImagefile = imagefile # save the name of the last image file read
2921	ParentFrame.oldImageTag = imagetag # save the tag of the last image file read
2922
2923	#################################################################################################
2924	# shortcut routines
2925	exp = np.exp
2926	sind = sin = s = lambda x: np.sin(x*np.pi/180.)
2927	cosd = cos = c = lambda x: np.cos(x*np.pi/180.)
2928	tand = tan = t = lambda x: np.tan(x*np.pi/180.)
2929	sqrt = sq = lambda x: np.sqrt(x)
2930	pi = lambda: np.pi
2931
2932	def FindFunction(f):
2933	'''Find the object corresponding to function f
2934
2935	:param str f: a function name such as 'numpy.exp'
2936	:returns: (pkgdict,pkgobj) where pkgdict contains a dict
2937	that defines the package location(s) and where pkgobj
2938	defines the object associated with the function.
2939	If the function is not found, pkgobj is None.
2940	'''
2941	df = f.split('.')
2942	pkgdict = {}
2943	# no listed module name, try in current namespace
2944	if len(df) == 1:
2945	try:
2946	fxnobj = eval(f)
2947	return pkgdict,fxnobj
2948	except (AttributeError, NameError):
2949	return None,None
2950
2951	# includes a package, see if package is already imported
2952	pkgnam = '.'.join(df[:-1])
2953	try:
2954	fxnobj = eval(f)
2955	pkgdict[pkgnam] = eval(pkgnam)
2956	return pkgdict,fxnobj
2957	except (AttributeError, NameError):
2958	pass
2959	# package not yet imported, so let's try
2960	if '.' not in sys.path: sys.path.append('.')
2961	pkgnam = '.'.join(df[:-1])
2962	#for pkg in f.split('.')[:-1]: # if needed, descend down the tree
2963	# if pkgname:
2964	# pkgname += '.' + pkg
2965	# else:
2966	# pkgname = pkg
2967	try:
2968	exec('import '+pkgnam)
2969	pkgdict[pkgnam] = eval(pkgnam)
2970	fxnobj = eval(f)
2971	except Exception as msg:
2972	print('load of '+pkgnam+' failed with error='+str(msg))
2973	return {},None
2974	# can we access the function? I am not exactly sure what
2975	# I intended this to test originally (BHT)
2976	try:
2977	fxnobj = eval(f,globals(),pkgdict)
2978	return pkgdict,fxnobj
2979	except Exception as msg:
2980	print('call to',f,' failed with error=',str(msg))
2981	return None,None # not found
2982
2983	class ExpressionObj(object):
2984	'''Defines an object with a user-defined expression, to be used for
2985	secondary fits or restraints. Object is created null, but is changed
2986	using :meth:`LoadExpression`. This contains only the minimum
2987	information that needs to be stored to save and load the expression
2988	and how it is mapped to GSAS-II variables.
2989	'''
2990	def __init__(self):
2991	self.expression = ''
2992	'The expression as a text string'
2993	self.assgnVars = {}
2994	'''A dict where keys are label names in the expression mapping to a GSAS-II
2995	variable. The value a G2 variable name.
2996	Note that the G2 variable name may contain a wild-card and correspond to
2997	multiple values.
2998	'''
2999	self.freeVars = {}
3000	'''A dict where keys are label names in the expression mapping to a free
3001	parameter. The value is a list with:
3002
3003	* a name assigned to the parameter
3004	* a value for to the parameter and
3005	* a flag to determine if the variable is refined.
3006	'''
3007	self.depVar = None
3008
3009	self.lastError = ('','')
3010	'''Shows last encountered error in processing expression
3011	(list of 1-3 str values)'''
3012
3013	self.distance_dict = None # to be used for defining atom phase/symmetry info
3014	self.distance_atoms = None # to be used for defining atom distances
3015
3016	def LoadExpression(self,expr,exprVarLst,varSelect,varName,varValue,varRefflag):
3017	'''Load the expression and associated settings into the object. Raises
3018	an exception if the expression is not parsed, if not all functions
3019	are defined or if not all needed parameter labels in the expression
3020	are defined.
3021
3022	This will not test if the variable referenced in these definitions
3023	are actually in the parameter dictionary. This is checked when the
3024	computation for the expression is done in :meth:`SetupCalc`.
3025
3026	:param str expr: the expression
3027	:param list exprVarLst: parameter labels found in the expression
3028	:param dict varSelect: this will be 0 for Free parameters
3029	and non-zero for expression labels linked to G2 variables.
3030	:param dict varName: Defines a name (str) associated with each free parameter
3031	:param dict varValue: Defines a value (float) associated with each free parameter
3032	:param dict varRefflag: Defines a refinement flag (bool)
3033	associated with each free parameter
3034	'''
3035	self.expression = expr
3036	self.compiledExpr = None
3037	self.freeVars = {}
3038	self.assgnVars = {}
3039	for v in exprVarLst:
3040	if varSelect[v] == 0:
3041	self.freeVars[v] = [
3042	varName.get(v),
3043	varValue.get(v),
3044	varRefflag.get(v),
3045	]
3046	else:
3047	self.assgnVars[v] = varName[v]
3048	self.CheckVars()
3049
3050	def EditExpression(self,exprVarLst,varSelect,varName,varValue,varRefflag):
3051	'''Load the expression and associated settings from the object into
3052	arrays used for editing.
3053
3054	:param list exprVarLst: parameter labels found in the expression
3055	:param dict varSelect: this will be 0 for Free parameters
3056	and non-zero for expression labels linked to G2 variables.
3057	:param dict varName: Defines a name (str) associated with each free parameter
3058	:param dict varValue: Defines a value (float) associated with each free parameter
3059	:param dict varRefflag: Defines a refinement flag (bool)
3060	associated with each free parameter
3061
3062	:returns: the expression as a str
3063	'''
3064	for v in self.freeVars:
3065	varSelect[v] = 0
3066	varName[v] = self.freeVars[v][0]
3067	varValue[v] = self.freeVars[v][1]
3068	varRefflag[v] = self.freeVars[v][2]
3069	for v in self.assgnVars:
3070	varSelect[v] = 1
3071	varName[v] = self.assgnVars[v]
3072	return self.expression
3073
3074	def GetVaried(self):
3075	'Returns the names of the free parameters that will be refined'
3076	return ["::"+self.freeVars[v][0] for v in self.freeVars if self.freeVars[v][2]]
3077
3078	def GetVariedVarVal(self):
3079	'Returns the names and values of the free parameters that will be refined'
3080	return [("::"+self.freeVars[v][0],self.freeVars[v][1]) for v in self.freeVars if self.freeVars[v][2]]
3081
3082	def UpdateVariedVars(self,varyList,values):
3083	'Updates values for the free parameters (after a refinement); only updates refined vars'
3084	for v in self.freeVars:
3085	if not self.freeVars[v][2]: continue
3086	if "::"+self.freeVars[v][0] not in varyList: continue
3087	indx = list(varyList).index("::"+self.freeVars[v][0])
3088	self.freeVars[v][1] = values[indx]
3089
3090	def GetIndependentVars(self):
3091	'Returns the names of the required independent parameters used in expression'
3092	return [self.assgnVars[v] for v in self.assgnVars]
3093
3094	def CheckVars(self):
3095	'''Check that the expression can be parsed, all functions are
3096	defined and that input loaded into the object is internally
3097	consistent. If not an Exception is raised.
3098
3099	:returns: a dict with references to packages needed to
3100	find functions referenced in the expression.
3101	'''
3102	ret = self.ParseExpression(self.expression)
3103	if not ret:
3104	raise Exception("Expression parse error")
3105	exprLblList,fxnpkgdict = ret
3106	# check each var used in expression is defined
3107	defined = list(self.assgnVars.keys()) + list(self.freeVars.keys())
3108	notfound = []
3109	for var in exprLblList:
3110	if var not in defined:
3111	notfound.append(var)
3112	if notfound:
3113	msg = 'Not all variables defined'
3114	msg1 = 'The following variables were not defined: '
3115	msg2 = ''
3116	for var in notfound:
3117	if msg: msg += ', '
3118	msg += var
3119	self.lastError = (msg1,' '+msg2)
3120	raise Exception(msg)
3121	return fxnpkgdict
3122
3123	def ParseExpression(self,expr):
3124	'''Parse an expression and return a dict of called functions and
3125	the variables used in the expression. Returns None in case an error
3126	is encountered. If packages are referenced in functions, they are loaded
3127	and the functions are looked up into the modules global
3128	workspace.
3129
3130	Note that no changes are made to the object other than
3131	saving an error message, so that this can be used for testing prior
3132	to the save.
3133
3134	:returns: a list of used variables
3135	'''
3136	self.lastError = ('','')
3137	import ast
3138	def ASTtransverse(node,fxn=False):
3139	'''Transverse a AST-parsed expresson, compiling a list of variables
3140	referenced in the expression. This routine is used recursively.
3141
3142	:returns: varlist,fxnlist where
3143	varlist is a list of referenced variable names and
3144	fxnlist is a list of used functions
3145	'''
3146	varlist = []
3147	fxnlist = []
3148	if isinstance(node, list):
3149	for b in node:
3150	v,f = ASTtransverse(b,fxn)
3151	varlist += v
3152	fxnlist += f
3153	elif isinstance(node, ast.AST):
3154	for a, b in ast.iter_fields(node):
3155	if isinstance(b, ast.AST):
3156	if a == 'func':
3157	fxnlist += ['.'.join(ASTtransverse(b,True)[0])]
3158	continue
3159	v,f = ASTtransverse(b,fxn)
3160	varlist += v
3161	fxnlist += f
3162	elif isinstance(b, list):
3163	v,f = ASTtransverse(b,fxn)
3164	varlist += v
3165	fxnlist += f
3166	elif node.__class__.__name__ == "Name":
3167	varlist += [b]
3168	elif fxn and node.__class__.__name__ == "Attribute":
3169	varlist += [b]
3170	return varlist,fxnlist
3171	try:
3172	exprast = ast.parse(expr)
3173	except SyntaxError:
3174	s = ''
3175	import traceback
3176	for i in traceback.format_exc().splitlines()[-3:-1]:
3177	if s: s += "\n"
3178	s += str(i)
3179	self.lastError = ("Error parsing expression:",s)
3180	return
3181	# find the variables & functions
3182	v,f = ASTtransverse(exprast)
3183	varlist = sorted(list(set(v)))
3184	fxnlist = list(set(f))
3185	pkgdict = {}
3186	# check the functions are defined
3187	for fxn in fxnlist:
3188	fxndict,fxnobj = FindFunction(fxn)
3189	if not fxnobj:
3190	self.lastError = ("Error: Invalid function",fxn,
3191	"is not defined")
3192	return
3193	if not hasattr(fxnobj,'__call__'):
3194	self.lastError = ("Error: Not a function.",fxn,
3195	"cannot be called as a function")
3196	return
3197	pkgdict.update(fxndict)
3198	return varlist,pkgdict
3199
3200	def GetDepVar(self):
3201	'return the dependent variable, or None'
3202	return self.depVar
3203
3204	def SetDepVar(self,var):
3205	'Set the dependent variable, if used'
3206	self.depVar = var
3207	#==========================================================================
3208	class ExpressionCalcObj(object):
3209	'''An object used to evaluate an expression from a :class:`ExpressionObj`
3210	object.
3211
3212	:param ExpressionObj exprObj: a :class:`~ExpressionObj` expression object with
3213	an expression string and mappings for the parameter labels in that object.
3214	'''
3215	def __init__(self,exprObj):
3216	self.eObj = exprObj
3217	'The expression and mappings; a :class:`ExpressionObj` object'
3218	self.compiledExpr = None
3219	'The expression as compiled byte-code'
3220	self.exprDict = {}
3221	'''dict that defines values for labels used in expression and packages
3222	referenced by functions
3223	'''
3224	self.lblLookup = {}
3225	'''Lookup table that specifies the expression label name that is
3226	tied to a particular GSAS-II parameters in the parmDict.
3227	'''
3228	self.fxnpkgdict = {}
3229	'''a dict with references to packages needed to
3230	find functions referenced in the expression.
3231	'''
3232	self.varLookup = {}
3233	'''Lookup table that specifies the GSAS-II variable(s)
3234	indexed by the expression label name. (Used for only for diagnostics
3235	not evaluation of expression.)
3236	'''
3237	self.su = None
3238	'''Standard error evaluation where supplied by the evaluator
3239	'''
3240	# Patch: for old-style expressions with a (now removed step size)
3241	if '2' in platform.python_version_tuple()[0]:
3242	basestr = basestring
3243	else:
3244	basestr = str
3245	for v in self.eObj.assgnVars:
3246	if not isinstance(self.eObj.assgnVars[v], basestr):
3247	self.eObj.assgnVars[v] = self.eObj.assgnVars[v][0]
3248	self.parmDict = {}
3249	'''A copy of the parameter dictionary, for distance and angle computation
3250	'''
3251
3252	def SetupCalc(self,parmDict):
3253	'''Do all preparations to use the expression for computation.
3254	Adds the free parameter values to the parameter dict (parmDict).
3255	'''
3256	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
3257	return
3258	self.fxnpkgdict = self.eObj.CheckVars()
3259	# all is OK, compile the expression
3260	self.compiledExpr = compile(self.eObj.expression,'','eval')
3261
3262	# look at first value in parmDict to determine its type
3263	parmsInList = True
3264	if '2' in platform.python_version_tuple()[0]:
3265	basestr = basestring
3266	else:
3267	basestr = str
3268	for key in parmDict:
3269	val = parmDict[key]
3270	if isinstance(val, basestr):
3271	parmsInList = False
3272	break
3273	try: # check if values are in lists
3274	val = parmDict[key][0]
3275	except (TypeError,IndexError):
3276	parmsInList = False
3277	break
3278
3279	# set up the dicts needed to speed computations
3280	self.exprDict = {}
3281	self.lblLookup = {}
3282	self.varLookup = {}
3283	for v in self.eObj.freeVars:
3284	varname = self.eObj.freeVars[v][0]
3285	varname = "::" + varname.lstrip(':').replace(' ','_').replace(':',';')
3286	self.lblLookup[varname] = v
3287	self.varLookup[v] = varname
3288	if parmsInList:
3289	parmDict[varname] = [self.eObj.freeVars[v][1],self.eObj.freeVars[v][2]]
3290	else:
3291	parmDict[varname] = self.eObj.freeVars[v][1]
3292	self.exprDict[v] = self.eObj.freeVars[v][1]
3293	for v in self.eObj.assgnVars:
3294	varname = self.eObj.assgnVars[v]
3295	if varname in parmDict:
3296	self.lblLookup[varname] = v
3297	self.varLookup[v] = varname
3298	if parmsInList:
3299	self.exprDict[v] = parmDict[varname][0]
3300	else:
3301	self.exprDict[v] = parmDict[varname]
3302	elif '*' in varname:
3303	varlist = LookupWildCard(varname,list(parmDict.keys()))
3304	if len(varlist) == 0:
3305	raise Exception("No variables match "+str(v))
3306	for var in varlist:
3307	self.lblLookup[var] = v
3308	if parmsInList:
3309	self.exprDict[v] = np.array([parmDict[var][0] for var in varlist])
3310	else:
3311	self.exprDict[v] = np.array([parmDict[var] for var in varlist])
3312	self.varLookup[v] = [var for var in varlist]
3313	else:
3314	self.exprDict[v] = None
3315	# raise Exception,"No value for variable "+str(v)
3316	self.exprDict.update(self.fxnpkgdict)
3317
3318	def UpdateVars(self,varList,valList):
3319	'''Update the dict for the expression with a set of values
3320	:param list varList: a list of variable names
3321	:param list valList: a list of corresponding values
3322	'''
3323	for var,val in zip(varList,valList):
3324	self.exprDict[self.lblLookup.get(var,'undefined: '+var)] = val
3325
3326	def UpdateDict(self,parmDict):
3327	'''Update the dict for the expression with values in a dict
3328	:param dict parmDict: a dict of values, items not in use are ignored
3329	'''
3330	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
3331	self.parmDict = parmDict
3332	return
3333	for var in parmDict:
3334	if var in self.lblLookup:
3335	self.exprDict[self.lblLookup[var]] = parmDict[var]
3336
3337	def EvalExpression(self):
3338	'''Evaluate an expression. Note that the expression
3339	and mapping are taken from the :class:`ExpressionObj` expression object
3340	and the parameter values were specified in :meth:`SetupCalc`.
3341	:returns: a single value for the expression. If parameter
3342	values are arrays (for example, from wild-carded variable names),
3343	the sum of the resulting expression is returned.
3344
3345	For example, if the expression is ``'A*B'``,
3346	where A is 2.0 and B maps to ``'1::Afrac:*'``, which evaluates to::
3347
3348	[0.5, 1, 0.5]
3349
3350	then the result will be ``4.0``.
3351	'''
3352	self.su = None
3353	if self.eObj.expression.startswith('Dist'):
3354	# GSASIIpath.IPyBreak()
3355	dist = G2mth.CalcDist(self.eObj.distance_dict, self.eObj.distance_atoms, self.parmDict)
3356	return dist
3357	elif self.eObj.expression.startswith('Angle'):
3358	angle = G2mth.CalcAngle(self.eObj.angle_dict, self.eObj.angle_atoms, self.parmDict)
3359	return angle
3360	if self.compiledExpr is None:
3361	raise Exception("EvalExpression called before SetupCalc")
3362	try:
3363	val = eval(self.compiledExpr,globals(),self.exprDict)
3364	except TypeError:
3365	val = None
3366	if not np.isscalar(val):
3367	val = np.sum(val)
3368	return val
3369
3370	class G2Exception(Exception):
3371	'A generic GSAS-II exception class'
3372	def __init__(self,msg):
3373	self.msg = msg
3374	def __str__(self):
3375	return repr(self.msg)
3376
3377	class G2RefineCancel(Exception):
3378	'Raised when Cancel is pressed in a refinement dialog'
3379	def __init__(self,msg):
3380	self.msg = msg
3381	def __str__(self):
3382	return repr(self.msg)
3383
3384	def HowDidIgetHere(wherecalledonly=False):
3385	'''Show a traceback with calls that brought us to the current location.
3386	Used for debugging.
3387	'''
3388	import traceback
3389	if wherecalledonly:
3390	i = traceback.format_list(traceback.extract_stack()[:-1])[-2]
3391	print(i.strip().rstrip())
3392	else:
3393	print (70'')
3394	for i in traceback.format_list(traceback.extract_stack()[:-1]): print(i.strip().rstrip())
3395	print (70'')
3396
3397	# Note that this is GUI code and should be moved at somepoint
3398	def CreatePDFitems(G2frame,PWDRtree,ElList,Qlimits,numAtm=1,FltBkg=0,PDFnames=[]):
3399	'''Create and initialize a new set of PDF tree entries
3400
3401	:param Frame G2frame: main GSAS-II tree frame object
3402	:param str PWDRtree: name of PWDR to be used to create PDF item
3403	:param dict ElList: data structure with composition
3404	:param list Qlimits: Q limits to be used for computing the PDF
3405	:param float numAtm: no. atom in chemical formula
3406	:param float FltBkg: flat background value
3407	:param list PDFnames: previously used PDF names
3408
3409	:returns: the Id of the newly created PDF entry
3410	'''
3411	PDFname = 'PDF '+PWDRtree[4:] # this places two spaces after PDF, which is needed is some places
3412	if PDFname in PDFnames:
3413	print('Skipping, entry already exists: '+PDFname)
3414	return None
3415	#PDFname = MakeUniqueLabel(PDFname,PDFnames)
3416	Id = G2frame.GPXtree.AppendItem(parent=G2frame.root,text=PDFname)
3417	Data = {
3418	'Sample':{'Name':PWDRtree,'Mult':1.0},
3419	'Sample Bkg.':{'Name':'','Mult':-1.0,'Refine':False},
3420	'Container':{'Name':'','Mult':-1.0,'Refine':False},
3421	'Container Bkg.':{'Name':'','Mult':-1.0},'ElList':ElList,
3422	'Geometry':'Cylinder','Diam':1.0,'Pack':0.50,'Form Vol':10.0*numAtm,'Flat Bkg':FltBkg,
3423	'DetType':'Area detector','ObliqCoeff':0.3,'Ruland':0.025,'QScaleLim':Qlimits,
3424	'Lorch':False,'BackRatio':0.0,'Rmax':100.,'noRing':False,'IofQmin':1.0,'Rmin':1.0,
3425	'I(Q)':[],'S(Q)':[],'F(Q)':[],'G(R)':[],
3426	#items for sequential PDFfit
3427	'Datarange':[0.,30.],'Fitrange':[0.,30.],'qdamp':[0.03,False],'qbroad':[0,False],'Temp':300}
3428	G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Controls'),Data)
3429	G2frame.GPXtree.SetItemPyData(G2frame.GPXtree.AppendItem(Id,text='PDF Peaks'),
3430	{'Limits':[1.,5.],'Background':[2,[0.,-0.2*np.pi],False],'Peaks':[]})
3431	return Id
3432
3433	class ShowTiming(object):
3434	'''An object to use for timing repeated sections of code.
3435
3436	Create the object with::
3437	tim0 = ShowTiming()
3438
3439	Tag sections of code to be timed with::
3440	tim0.start('start')
3441	tim0.start('in section 1')
3442	tim0.start('in section 2')
3443
3444	etc. (Note that each section should have a unique label.)
3445
3446	After the last section, end timing with::
3447	tim0.end()
3448
3449	Show timing results with::
3450	tim0.show()
3451
3452	'''
3453	def __init__(self):
3454	self.timeSum = []
3455	self.timeStart = []
3456	self.label = []
3457	self.prev = None
3458	def start(self,label):
3459	import time
3460	if label in self.label:
3461	i = self.label.index(label)
3462	self.timeStart[i] = time.time()
3463	else:
3464	i = len(self.label)
3465	self.timeSum.append(0.0)
3466	self.timeStart.append(time.time())
3467	self.label.append(label)
3468	if self.prev is not None:
3469	self.timeSum[self.prev] += self.timeStart[i] - self.timeStart[self.prev]
3470	self.prev = i
3471	def end(self):
3472	import time
3473	if self.prev is not None:
3474	self.timeSum[self.prev] += time.time() - self.timeStart[self.prev]
3475	self.prev = None
3476	def show(self):
3477	sumT = sum(self.timeSum)
3478	print('Timing results (total={:.2f} sec)'.format(sumT))
3479	for i,(lbl,val) in enumerate(zip(self.label,self.timeSum)):
3480	print('{} {:20} {:8.2f} ms {:5.2f}%'.format(i,lbl,1000.val,100val/sumT))
3481
3482	def validateAtomDrawType(typ,generalData={}):
3483	'''Confirm that the selected Atom drawing type is valid for the current
3484	phase. If not, use 'vdW balls'. This is currently used only for setting a
3485	default when atoms are added to the atoms draw list.
3486	'''
3487	if typ in ('lines','vdW balls','sticks','balls & sticks','ellipsoids'):
3488	return typ
3489	# elif generalData.get('Type','') == 'macromolecular':
3490	# if typ in ('backbone',):
3491	# return typ
3492	return 'vdW balls'
3493
3494	if __name__ == "__main__":
3495	# test variable descriptions
3496	for var in '0::Afrac:*',':1:Scale','1::dAx:0','::undefined':
3497	v = var.split(':')[2]
3498	print(var+':\t', getDescr(v),getVarStep(v))
3499	import sys; sys.exit()
3500	# test equation evaluation
3501	def showEQ(calcobj):
3502	print (50*'=')
3503	print (calcobj.eObj.expression+'='+calcobj.EvalExpression())
3504	for v in sorted(calcobj.varLookup):
3505	print (" "+v+'='+calcobj.exprDict[v]+'='+calcobj.varLookup[v])
3506	# print ' Derivatives'
3507	# for v in calcobj.derivStep.keys():
3508	# print ' d(Expr)/d('+v+') =',calcobj.EvalDeriv(v)
3509
3510	obj = ExpressionObj()
3511
3512	obj.expression = "A*np.exp(B)"
3513	obj.assgnVars = {'B': '0::Afrac:1'}
3514	obj.freeVars = {'A': [u'A', 0.5, True]}
3515	#obj.CheckVars()
3516	calcobj = ExpressionCalcObj(obj)
3517
3518	obj1 = ExpressionObj()
3519	obj1.expression = "A*np.exp(B)"
3520	obj1.assgnVars = {'B': '0::Afrac:*'}
3521	obj1.freeVars = {'A': [u'Free Prm A', 0.5, True]}
3522	#obj.CheckVars()
3523	calcobj1 = ExpressionCalcObj(obj1)
3524
3525	obj2 = ExpressionObj()
3526	obj2.distance_stuff = np.array([[0,1],[1,-1]])
3527	obj2.expression = "Dist(1,2)"
3528	GSASIIpath.InvokeDebugOpts()
3529	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
3530	calcobj2 = ExpressionCalcObj(obj2)
3531	calcobj2.SetupCalc(parmDict2)
3532	showEQ(calcobj2)
3533
3534	parmDict1 = {'0::Afrac:0':1.0, '0::Afrac:1': 1.0}
3535	print ('\nDict = '+parmDict1)
3536	calcobj.SetupCalc(parmDict1)
3537	showEQ(calcobj)
3538	calcobj1.SetupCalc(parmDict1)
3539	showEQ(calcobj1)
3540
3541	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
3542	print ('Dict = '+parmDict2)
3543	calcobj.SetupCalc(parmDict2)
3544	showEQ(calcobj)
3545	calcobj1.SetupCalc(parmDict2)
3546	showEQ(calcobj1)
3547	calcobj2.SetupCalc(parmDict2)
3548	showEQ(calcobj2)

Note: See TracBrowser for help on using the repository browser.

Download in other formats: