Context Navigation

source: trunk/GSASIIobj.py @ 2845

Last change on this file since 2845 was 2845, checked in by toby, 6 years ago
fix for readthedocs (import without full set of packages
Property svn:eol-style set to `native` Property svn:keywords set to `Date Author Revision URL Id`
File size: 124.9 KB

Line
1	# -- coding: utf-8 --
2	#GSASIIobj - data objects for GSAS-II
3	########### SVN repository information ###################
4	# $Date: 2017-05-30 21:26:42 +0000 (Tue, 30 May 2017) $
5	# $Author: toby $
6	# $Revision: 2845 $
7	# $URL: trunk/GSASIIobj.py $
8	# $Id: GSASIIobj.py 2845 2017-05-30 21:26:42Z toby $
9	########### SVN repository information ###################
10
11	'''
12	GSASIIobj: Data objects
13	=========================
14
15	This module defines and/or documents the data structures used in GSAS-II, as well
16	as provides misc. support routines.
17
18	Constraints Tree Item
19	----------------------
20
21	.. _Constraints_table:
22
23	.. index::
24	single: Constraints object description
25	single: Data object descriptions; Constraints
26
27	Constraints are stored in a dict, separated into groups.
28	Note that parameter are named in the following pattern,
29	p:h:<var>:n, where p is the phase number, h is the histogram number
30	<var> is a variable name and n is the parameter number.
31	If a parameter does not depend on a histogram or phase or is unnumbered, that
32	number is omitted.
33	Note that the contents of each dict item is a List where each element in the
34	list is a :ref:`constraint definition objects <Constraint_definitions_table>`.
35	The constraints in this form are converted in
36	:func:`GSASIIstrIO.ProcessConstraints` to the form used in :mod:`GSASIImapvars`
37
38	The keys in the Constraints dict are:
39
40	.. tabularcolumns:: \|l\|p{4.5in}\|
41
42	========== ====================================================
43	key explanation
44	========== ====================================================
45	Hist This specifies a list of constraints on
46	histogram-related parameters,
47	which will be of form :h:<var>:n.
48	HAP This specifies a list of constraints on parameters
49	that are defined for every histogram in each phase
50	and are of form p:h:<var>:n.
51	Phase This specifies a list of constraints on phase
52	parameters,
53	which will be of form p::<var>:n.
54	Global This specifies a list of constraints on parameters
55	that are not tied to a histogram or phase and
56	are of form ::<var>:n
57	========== ====================================================
58
59	.. _Constraint_definitions_table:
60
61	.. index::
62	single: Constraint definition object description
63	single: Data object descriptions; Constraint Definition
64
65	Each constraint is defined as an item in a list. Each constraint is of form::
66
67	[[<mult1>, <var1>], [<mult2>, <var2>],..., <fixedval>, <varyflag>, <constype>]
68
69	Where the variable pair list item containing two values [<mult>, <var>], where:
70
71	* <mult> is a multiplier for the constraint (float)
72	* <var> a :class:`G2VarObj` object (previously a str variable name of form
73	'p:h:name[:at]')
74
75	Note that the last three items in the list play a special role:
76
77	* <fixedval> is the fixed value for a `constant equation` (``constype=c``)
78	constraint or is None. For a `New variable` (``constype=f``) constraint,
79	a variable name can be specified as a str (used for externally
80	generated constraints)
81	* <varyflag> is True or False for `New variable` (``constype=f``) constraints
82	or is None. This will be implemented in the future to indicate if these variables
83	should be refined.
84	* <constype> is one of four letters, 'e', 'c', 'h', 'f' that determines the type of constraint:
85
86	* 'e' defines a set of equivalent variables. Only the first variable is refined (if the
87	appropriate refine flag is set) and and all other equivalent variables in the list
88	are generated from that variable, using the appropriate multipliers.
89	* 'c' defines a constraint equation of form,
90	:math:`m_1 \\times var_1 + m_2 \\times var_2 + ... = c`
91	* 'h' defines a variable to hold (not vary). Any variable on this list is not varied,
92	even if its refinement flag is set. Only one [mult,var] pair is allowed in a hold
93	constraint and the mult value is ignored.
94	This is of particular value when needing to hold one or more variables where a
95	single flag controls a set of variables such as, coordinates,
96	the reciprocal metric tensor or anisotropic displacement parameter.
97	* 'f' defines a new variable (function) according to relationship
98	:math:`newvar = m_1 \\times var_1 + m_2 \\times var_2 + ...`
99
100	Covariance Tree Item
101	--------------------
102
103	.. _Covariance_table:
104
105	.. index::
106	single: Covariance description
107	single: Data object descriptions; Covariance
108
109	The Covariance tree item has results from the last least-squares run. They
110	are stored in a dict with these keys:
111
112	.. tabularcolumns:: \|l\|l\|p{4in}\|
113
114	============= =============== ====================================================
115	key sub-key explanation
116	============= =============== ====================================================
117	newCellDict \ dict with lattice parameters computed by
118	:func:`GSASIIstrMath.GetNewCellParms` (dict)
119	title \ Name of gpx file(?) (str)
120	variables \ Values for all N refined variables
121	(list of float values, length N,
122	ordered to match varyList)
123	sig \ Uncertainty values for all N refined variables
124	(list of float values, length N,
125	ordered to match varyList)
126	varyList \ List of directly refined variables
127	(list of str values, length N)
128	newAtomDict \ dict with atom position values computed in
129	:func:`GSASIIstrMath.ApplyXYZshifts` (dict)
130	Rvals \ R-factors, GOF, Marquardt value for last
131	refinement cycle (dict)
132	\ Nobs Number of observed data points (int)
133	\ Rwp overall weighted profile R-factor (%, float)
134	\ chisq sum[w(Iobs-Icalc)*2] for all data
135	note this is not the reduced chi squared (float)
136	\ lamMax Marquardt value applied to Hessian diagonal
137	(float)
138	\ GOF The goodness-of-fit, aka square root of
139	the reduced chi squared. (float)
140	covMatrix \ The (NxN) covVariance matrix (np.array)
141	============= =============== ====================================================
142
143	Phase Tree Items
144	----------------
145
146	.. _Phase_table:
147
148	.. index::
149	single: Phase object description
150	single: Data object descriptions; Phase
151
152	Phase information is stored in the GSAS-II data tree as children of the
153	Phases item in a dict with keys:
154
155	.. tabularcolumns:: \|l\|l\|p{4in}\|
156
157	========== =============== ====================================================
158	key sub-key explanation
159	========== =============== ====================================================
160	General \ Overall information for the phase (dict)
161	\ AtomPtrs list of four locations to use to pull info
162	from the atom records (list)
163	\ F000X x-ray F(000) intensity (float)
164	\ F000N neutron F(000) intensity (float)
165	\ Mydir directory of current .gpx file (str)
166	\ MCSA controls Monte Carlo-Simulated Annealing controls (dict)
167	\ Cell List with 8 items: cell refinement flag (bool)
168	a, b, c, (Angstrom, float)
169	alpha, beta & gamma (degrees, float)
170	volume (A^3, float)
171	\ Type 'nuclear' or 'macromolecular' for now (str)
172	\ Map dict of map parameters
173	\ SH Texture dict of spherical harmonic preferred orientation
174	parameters
175	\ Isotope dict of isotopes for each atom type
176	\ Isotopes dict of scattering lengths for each isotope
177	combination for each element in phase
178	\ Name phase name (str)
179	\ SGData Space group details as a :ref:`space group (SGData) object <SGData_table>`
180	as defined in :func:`GSASIIspc.SpcGroup`.
181	\ Pawley neg wt Restraint value for negative Pawley intensities
182	(float)
183	\ Flip dict of Charge flip controls
184	\ Data plot type data plot type ('Mustrain', 'Size' or
185	'Preferred orientation') for powder data (str)
186	\ Mass Mass of unit cell contents in g/mol
187	\ POhkl March-Dollase preferred orientation direction
188	\ Z dict of atomic numbers for each atom type
189	\ vdWRadii dict of van der Waals radii for each atom type
190	\ Color Colors for atoms (list of (r,b,g) triplets)
191	\ AtomTypes List of atom types
192	\ AtomMass List of masses for atoms
193	\ doPawley Flag for Pawley intensity extraction (bool)
194	\ NoAtoms Number of atoms per unit cell of each type (dict)
195	\ Pawley dmin maximum Q (as d-space) to use for Pawley
196	extraction (float)
197	\ BondRadii Default radius for each atom used to compute
198	interatomic distances (list of floats)
199	\ AngleRadii Default radius for each atom used to compute
200	interatomic angles (list of floats)
201	\ DisAglCtls Dict with distance/angle search controls,
202	which has keys 'Name', 'AtomTypes',
203	'BondRadii', 'AngleRadii' which are as above
204	except are possibly edited. Also contains
205	'Factors', which is a 2 element list with
206	a multiplier for bond and angle search range
207	[typically (0.85,0.85)].
208	ranId \ unique random number Id for phase (int)
209	pId \ Phase Id number for current project (int).
210	Atoms \ Atoms in phase as a list of lists. The outer list
211	is for each atom, the inner list contains varying
212	items depending on the type of phase, see
213	the :ref:`Atom Records <Atoms_table>` description.
214	(list of lists)
215	Drawing \ Display parameters (dict)
216	\ ballScale Size of spheres in ball-and-stick display (float)
217	\ bondList dict with bonds
218	\ contourLevel map contour level in e/A^3 (float)
219	\ showABC Flag to show view point triplet (bool). True=show.
220	\ viewDir cartesian viewing direction (np.array with three
221	elements)
222	\ Zclip clipping distance in A (float)
223	\ backColor background for plot as and R,G,B triplet
224	(default = [0, 0, 0], black).
225	(list with three atoms)
226	\ selectedAtoms List of selected atoms (list of int values)
227	\ showRigidBodies Flag to highlight rigid body placement
228	\ sizeH Size ratio for H atoms (float)
229	\ bondRadius Size of binds in A (float)
230	\ atomPtrs positions of x, type, site sym, ADP flag in Draw Atoms (list)
231	\ viewPoint list of lists. First item in list is [x,y,z]
232	in fractional coordinates for the center of
233	the plot. Second item list of previous & current
234	atom number viewed (may be [0,0])
235	\ showHydrogen Flag to control plotting of H atoms.
236	\ unitCellBox Flag to control display of the unit cell.
237	\ ellipseProb Probability limit for display of thermal
238	ellipsoids in % (float).
239	\ vdwScale Multiplier of van der Waals radius for
240	display of vdW spheres.
241	\ Atoms A list of lists with an entry for each atom
242	that is plotted.
243	\ Zstep Step to de/increase Z-clip (float)
244	\ Quaternion Viewing quaternion (4 element np.array)
245	\ radiusFactor Distance ratio for searching for bonds. ? Bonds
246	are located that are within r(Ra+Rb) and (Ra+Rb)/r
247	where Ra and Rb are the atomic radii.
248	\ oldxy previous view point (list with two floats)
249	\ cameraPos Viewing position in A for plot (float)
250	\ depthFog True if use depthFog on plot - set currently as False (bool)
251	RBModels \ Rigid body assignments (note Rigid body definitions
252	are stored in their own main top-level tree entry.)
253	Pawley ref \ Pawley reflections
254	Histograms \ A dict of dicts. The key for the outer dict is
255	the histograms tied to this phase. The inner
256	dict contains the combined phase/histogram
257	parameters for items such as scale factors,
258	size and strain parameters. (dict)
259	MCSA \ Monte-Carlo simulated annealing parameters (dict)
260	\
261	========== =============== ====================================================
262
263	Rigid Body Objects
264	------------------
265
266	.. _RBData_table:
267
268	.. index::
269	single: Rigid Body Data description
270	single: Data object descriptions; Rigid Body Data
271
272	Rigid body descriptions are available for two types of rigid bodies: 'Vector'
273	and 'Residue'. Vector rigid bodies are developed by a sequence of translations each
274	with a refinable magnitude and Residue rigid bodies are described as Cartesian coordinates
275	with defined refinable torsion angles.
276
277	.. tabularcolumns:: \|l\|l\|p{4in}\|
278
279	========== =============== ====================================================
280	key sub-key explanation
281	========== =============== ====================================================
282	Vector RBId vector rigid bodies (dict of dict)
283	\ AtInfo Drad, Color: atom drawing radius & color for each atom type (dict)
284	\ RBname Name assigned by user to rigid body (str)
285	\ VectMag vector magnitudes in A (list)
286	\ rbXYZ Cartesian coordinates for Vector rigid body (list of 3 float)
287	\ rbRef 3 assigned reference atom nos. in rigid body for origin
288	definition, use center of atoms flag (list of 3 int & 1 bool)
289	\ VectRef refinement flags for VectMag values (list of bool)
290	\ rbTypes Atom types for each atom in rigid body (list of str)
291	\ rbVect Cartesian vectors for each translation used to build rigid body (list of lists)
292	\ useCount Number of times rigid body is used in any structure (int)
293	Residue RBId residue rigid bodies (dict of dict)
294	\ AtInfo Drad, Color: atom drawing radius & color for each atom type(dict)
295	\ RBname Name assigned by user to rigid body (str)
296	\ rbXYZ Cartesian coordinates for Residue rigid body (list of 3 float)
297	\ rbTypes Atom types for each atom in rigid body (list of str)
298	\ atNames Names of each atom in rigid body (e.g. C1,N2...) (list of str)
299	\ rbRef 3 assigned reference atom nos. in rigid body for origin
300	definition, use center of atoms flag (list of 3 int & 1 bool)
301	\ rbSeq Orig,Piv,angle,Riding (list): definition of internal rigid body
302	torsion; origin atom (int), pivot atom (int), torsion angle (float),
303	riding atoms (list of int)
304	\ SelSeq [int,int] used by SeqSizer to identify objects
305	\ useCount Number of times rigid body is used in any structure (int)
306	RBIds \ unique Ids generated upon creation of each rigid body (dict)
307	\ Vector Ids for each Vector rigid body (list)
308	\ Residue Ids for each Residue rigid body (list)
309	========== =============== ====================================================
310
311	Space Group Objects
312	-------------------
313
314	.. _SGData_table:
315
316	.. index::
317	single: Space Group Data description
318	single: Data object descriptions; Space Group Data
319
320	Space groups are interpreted by :func:`GSASIIspc.SpcGroup`
321	and the information is placed in a SGdata object
322	which is a dict with these keys:
323
324	.. tabularcolumns:: \|l\|p{4.5in}\|
325
326	========== ====================================================
327	key explanation
328	========== ====================================================
329	SpGrp space group symbol (str)
330	Laue one of the following 14 Laue classes:
331	-1, 2/m, mmm, 4/m, 4/mmm, 3R,
332	3mR, 3, 3m1, 31m, 6/m, 6/mmm, m3, m3m (str)
333	SGInv True if centrosymmetric, False if not (bool)
334	SGLatt Lattice centering type. Will be one of
335	P, A, B, C, I, F, R (str)
336	SGUniq unique axis if monoclinic. Will be
337	a, b, or c for monoclinic space groups.
338	Will be blank for non-monoclinic. (str)
339	SGCen Symmetry cell centering vectors. A (n,3) np.array
340	of centers. Will always have at least one row:
341	``np.array([[0, 0, 0]])``
342	SGOps symmetry operations as a list of form
343	``[[M1,T1], [M2,T2],...]``
344	where :math:`M_n` is a 3x3 np.array
345	and :math:`T_n` is a length 3 np.array.
346	Atom coordinates are transformed where the
347	Asymmetric unit coordinates [X is (x,y,z)]
348	are transformed using
349	:math:`X^\prime = M_n*X+T_n`
350	SGSys symmetry unit cell: type one of
351	'triclinic', 'monoclinic', 'orthorhombic',
352	'tetragonal', 'rhombohedral', 'trigonal',
353	'hexagonal', 'cubic' (str)
354	SGPolax Axes for space group polarity. Will be one of
355	'', 'x', 'y', 'x y', 'z', 'x z', 'y z',
356	'xyz'. In the case where axes are arbitrary
357	'111' is used (P 1, and ?).
358	========== ====================================================
359
360	.. _SSGData_table:
361
362	.. index::
363	single: Superspace Group Data description
364	single: Data object descriptions; Superspace Group Data
365
366	Superspace groups [3+1] are interpreted by :func:`GSASIIspc.SSpcGroup`
367	and the information is placed in a SSGdata object
368	which is a dict with these keys:
369
370	.. tabularcolumns:: \|l\|p{4.5in}\|
371
372	========== ====================================================
373	key explanation
374	========== ====================================================
375	SSpGrp superspace group symbol extension to space group
376	symbol, accidental spaces removed
377	SSGCen 4D cell centering vectors [0,0,0,0] at least
378	SSGOps 4D symmetry operations as [M,T] so that M*x+T = x'
379	========== ====================================================
380
381
382	Atom Records
383	------------
384
385	.. _Atoms_table:
386
387	.. index::
388	single: Atoms record description
389	single: Data object descriptions; Atoms record
390
391
392	If ``phasedict`` points to the phase information in the data tree, then
393	atoms are contained in a list of atom records (list) in
394	``phasedict['Atoms']``. Also needed to read atom information
395	are four pointers, ``cx,ct,cs,cia = phasedict['General']['atomPtrs']``,
396	which define locations in the atom record, as shown below. Items shown are
397	always present; additional ones for macromolecular phases are marked 'mm'
398
399	.. tabularcolumns:: \|l\|p{4.5in}\|
400
401	============== ====================================================
402	location explanation
403	============== ====================================================
404	ct-4 mm - residue number (str)
405	ct-3 mm - residue name (e.g. ALA) (str)
406	ct-2 mm - chain label (str)
407	ct-1 atom label (str)
408	ct atom type (str)
409	ct+1 refinement flags; combination of 'F', 'X', 'U' (str)
410	cx,cx+1,cx+2 the x,y and z coordinates (3 floats)
411	cs site symmetry (str)
412	cs+1 site multiplicity (int)
413	cia ADP flag: Isotropic ('I') or Anisotropic ('A')
414	cia+1 Uiso (float)
415	cia+2...cia+7 U11, U22, U33, U12, U13, U23 (6 floats)
416	atom[cia+8] unique atom identifier (int)
417
418	============== ====================================================
419
420	Drawing Atom Records
421	--------------------
422
423	.. _Drawing_atoms_table:
424
425	.. index::
426	single: Drawing atoms record description
427	single: Data object descriptions; Drawing atoms record
428
429
430	If ``phasedict`` points to the phase information in the data tree, then
431	drawing atoms are contained in a list of drawing atom records (list) in
432	``phasedict['Drawing']['Atoms']``. Also needed to read atom information
433	are four pointers, ``cx,ct,cs,ci = phasedict['Drawing']['AtomPtrs']``,
434	which define locations in the atom record, as shown below. Items shown are
435	always present; additional ones for macromolecular phases are marked 'mm'
436
437	.. tabularcolumns:: \|l\|p{4.5in}\|
438
439	============== ====================================================
440	location explanation
441	============== ====================================================
442	ct-4 mm - residue number (str)
443	ct-3 mm - residue name (e.g. ALA) (str)
444	ct-2 mm - chain label (str)
445	ct-1 atom label (str)
446	ct atom type (str)
447	cx,cx+1,cx+2 the x,y and z coordinates (3 floats)
448	cs-1 Sym Op symbol; sym. op number + unit cell id (e.g. '1,0,-1') (str)
449	cs atom drawing style; e.g. 'balls & sticks' (str)
450	cs+1 atom label style (e.g. 'name') (str)
451	cs+2 atom color (RBG triplet) (int)
452	cs+3 ADP flag: Isotropic ('I') or Anisotropic ('A')
453	cs+4 Uiso (float)
454	cs+5...cs+11 U11, U22, U33, U12, U13, U23 (6 floats)
455	ci unique atom identifier; matches source atom Id in Atom Records (int)
456	============== ====================================================
457
458	Powder Diffraction Tree Items
459	-----------------------------
460
461	.. _Powder_table:
462
463	.. index::
464	single: Powder data object description
465	single: Data object descriptions; Powder Data
466
467	Every powder diffraction histogram is stored in the GSAS-II data tree
468	with a top-level entry named beginning with the string "PWDR ". The
469	diffraction data for that information are directly associated with
470	that tree item and there are a series of children to that item. The
471	routines :func:`GSASII.GSASII.GetUsedHistogramsAndPhasesfromTree`
472	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
473	load this information into a dictionary where the child tree name is
474	used as a key, and the information in the main entry is assigned
475	a key of ``Data``, as outlined below.
476
477	.. tabularcolumns:: \|p{1in}\|p{1in}\|p{4in}\|
478
479	====================== =============== ====================================================
480	key sub-key explanation
481	====================== =============== ====================================================
482	Comments \ Text strings extracted from the original powder
483	data header. These cannot be changed by the user;
484	it may be empty.
485	Limits \ A list of two two element lists, as [[Ld,Hd],[L,H]]
486	where L and Ld are the current and default lowest
487	two-theta value to be used and
488	where H and Hd are the current and default highest
489	two-theta value to be used.
490	Reflection Lists \ A dict with an entry for each phase in the
491	histogram. The contents of each dict item
492	is a dict containing reflections, as described in
493	the :ref:`Powder Reflections <PowderRefl_table>`
494	description.
495	Instrument Parameters \ A list containing two dicts where the possible
496	keys in each dict are listed below. The value
497	for each item is a list containing three values:
498	the initial value, the current value and a
499	refinement flag which can have a value of
500	True, False or 0 where 0 indicates a value that
501	cannot be refined. The first and second
502	values are floats unless otherwise noted.
503	Items in the first dict are noted as [1]
504	\ Lam Specifies a wavelength in Angstroms [1]
505	\ Lam1 Specifies the primary wavelength in
506	Angstrom, when an alpha1, alpha2
507	source is used [1]
508	\ Lam2 Specifies the secondary wavelength in
509	Angstrom, when an alpha1, alpha2
510	source is used [1]
511	I(L2)/I(L1) Ratio of Lam2 to Lam1 [1]
512	\ Type Histogram type (str) [1]:
513	* 'PXC' for constant wavelength x-ray
514	* 'PNC' for constant wavelength neutron
515	* 'PNT' for time of flight neutron
516	\ Zero Two-theta zero correction in degrees [1]
517	\ Azimuth Azimuthal setting angle for data recorded
518	with differing setting angles [1]
519	\ U, V, W Cagliotti profile coefficients
520	for Gaussian instrumental broadening, where the
521	FWHM goes as
522	:math:`U \\tan^2\\theta + V \\tan\\theta + W` [1]
523	\ X, Y Cauchy (Lorentzian) instrumental broadening
524	coefficients [1]
525	\ SH/L Variant of the Finger-Cox-Jephcoat asymmetric
526	peak broadening ratio. Note that this is the
527	average between S/L and H/L where S is
528	sample height, H is the slit height and
529	L is the goniometer diameter. [1]
530	\ Polariz. Polarization coefficient. [1]
531	wtFactor \ A weighting factor to increase or decrease
532	the leverage of data in the histogram (float).
533	A value of 1.0 weights the data with their
534	standard uncertainties and a larger value
535	increases the weighting of the data (equivalent
536	to decreasing the uncertainties).
537	Sample Parameters \ Specifies a dict with parameters that describe how
538	the data were collected, as listed
539	below. Refinable parameters are a list containing
540	a float and a bool, where the second value
541	specifies if the value is refined, otherwise
542	the value is a float unless otherwise noted.
543	\ Scale The histogram scale factor (refinable)
544	\ Absorption The sample absorption coefficient as
545	:math:`\\mu r` where r is the radius
546	(refinable). Only valid for Debye-Scherrer geometry.
547	\ SurfaceRoughA Surface roughness parameter A as defined by
548	Surotti,J. Appl. Cryst, 5,325-331, 1972.(refinable -
549	only valid for Bragg-Brentano geometry)
550	\ SurfaceRoughB Surface roughness parameter B (refinable -
551	only valid for Bragg-Brentano geometry)
552	\ DisplaceX, Sample displacement from goniometer center
553	DisplaceY where Y is along the beam direction and
554	X is perpendicular. Units are :math:`\\mu m`
555	(refinable).
556	\ Phi, Chi, Goniometer sample setting angles, in degrees.
557	Omega
558	\ Gonio. radius Radius of the diffractometer in mm
559	\ InstrName A name for the instrument, used in preparing
560	a CIF (str).
561	\ Force, Variables that describe how the measurement
562	Temperature, was performed. Not used directly in
563	Humidity, any computations.
564	Pressure,
565	Voltage
566	\ ranId The random-number Id for the histogram
567	(same value as where top-level key is ranId)
568	\ Type Type of diffraction data, may be 'Debye-Scherrer'
569	or 'Bragg-Brentano' (str).
570	\ Diffuse not in use?
571	hId \ The number assigned to the histogram when
572	the project is loaded or edited (can change)
573	ranId \ A random number id for the histogram
574	that does not change
575	Background \ The background is stored as a list with where
576	the first item in the list is list and the second
577	item is a dict. The list contains the background
578	function and its coefficients; the dict contains
579	Debye diffuse terms and background peaks.
580	(TODO: this needs to be expanded.)
581	Data \ The data consist of a list of 6 np.arrays
582	containing in order:
583
584	0. the x-postions (two-theta in degrees),
585	1. the intensity values (Yobs),
586	2. the weights for each Yobs value
587	3. the computed intensity values (Ycalc)
588	4. the background values
589	5. Yobs-Ycalc
590	====================== =============== ====================================================
591
592	Powder Reflection Data Structure
593	--------------------------------
594
595	.. _PowderRefl_table:
596
597	.. index::
598	single: Powder reflection object description
599	single: Data object descriptions; Powder Reflections
600
601	For every phase in a histogram, the ``Reflection Lists`` value is a dict
602	one element of which is `'RefList'`, which is a np.array containing
603	reflections. The columns in that array are documented below.
604
605	========== ====================================================
606	index explanation
607	========== ====================================================
608	0,1,2 h,k,l (float)
609	3 multiplicity
610	4 d-space, Angstrom
611	5 pos, two-theta
612	6 sig, Gaussian width
613	7 gam, Lorenzian width
614	8 :math:`F_{obs}^2`
615	9 :math:`F_{calc}^2`
616	10 reflection phase, in degrees
617	11 intensity correction for reflection, this times
618	:math:`F_{obs}^2` or :math:`F_{calc}^2` gives Iobs or Icalc
619	========== ====================================================
620
621	Single Crystal Tree Items
622	-------------------------
623
624	.. _Xtal_table:
625
626	.. index::
627	single: Single Crystal data object description
628	single: Data object descriptions; Single crystal data
629
630	Every single crystal diffraction histogram is stored in the GSAS-II data tree
631	with a top-level entry named beginning with the string "HKLF ". The
632	diffraction data for that information are directly associated with
633	that tree item and there are a series of children to that item. The
634	routines :func:`GSASII.GSASII.GetUsedHistogramsAndPhasesfromTree`
635	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
636	load this information into a dictionary where the child tree name is
637	used as a key, and the information in the main entry is assigned
638	a key of ``Data``, as outlined below.
639
640	.. tabularcolumns:: \|l\|l\|p{4in}\|
641
642	====================== =============== ====================================================
643	key sub-key explanation
644	====================== =============== ====================================================
645	Data \ A dict that contains the
646	reflection table,
647	as described in the
648	:ref:`Single Crystal Reflections
649	<XtalRefl_table>`
650	description.
651
652	Instrument Parameters \ A list containing two dicts where the possible
653	keys in each dict are listed below. The value
654	for most items is a list containing two values:
655	the initial value, the current value.
656	The first and second
657	values are floats unless otherwise noted.
658	\ Lam Specifies a wavelength in Angstroms (two floats)
659	\ Type Histogram type (two str values):
660	* 'SXC' for constant wavelength x-ray
661	* 'SNC' for constant wavelength neutron
662	* 'SNT' for time of flight neutron
663	\ InstrName A name for the instrument, used in preparing
664	a CIF (str).
665
666	wtFactor \ A weighting factor to increase or decrease
667	the leverage of data in the histogram (float).
668	A value of 1.0 weights the data with their
669	standard uncertainties and a larger value
670	increases the weighting of the data (equivalent
671	to decreasing the uncertainties).
672
673	hId \ The number assigned to the histogram when
674	the project is loaded or edited (can change)
675	ranId \ A random number id for the histogram
676	that does not change
677	====================== =============== ====================================================
678
679	Single Crystal Reflection Data Structure
680	----------------------------------------
681
682	.. _XtalRefl_table:
683
684	.. index::
685	single: Single Crystal reflection object description
686	single: Data object descriptions; Single Crystal Reflections
687
688	For every single crystal a histogram, the ``'Data'`` item contains
689	the structure factors as an np.array in item `'RefList'`.
690	The columns in that array are documented below.
691
692	========== ====================================================
693	index explanation
694	========== ====================================================
695	0,1,2 h,k,l (float)
696	3 multiplicity
697	4 d-space, Angstrom
698	5 :math:`F_{obs}^2`
699	6 :math:`\sigma(F_{obs}^2)`
700	7 :math:`F_{calc}^2`
701	8 :math:`F_{obs}^2T`
702	9 :math:`F_{calc}^2T`
703	10 reflection phase, in degrees
704	11 intensity correction for reflection, this times
705	:math:`F_{obs}^2` or :math:`F_{calc}^2`
706	gives Iobs or Icalc
707	========== ====================================================
708
709	Image Data Structure
710	--------------------
711
712	.. _Image_table:
713
714	.. index::
715	image: Image data object description
716	image: Image object descriptions
717
718	Every 2-dimensional image is stored in the GSAS-II data tree
719	with a top-level entry named beginning with the string "IMG ". The
720	image data are directly associated with that tree item and there
721	are a series of children to that item. The routines :func:`GSASII.GSASII.GetUsedHistogramsAndPhasesfromTree`
722	and :func:`GSASIIstrIO.GetUsedHistogramsAndPhases` will
723	load this information into a dictionary where the child tree name is
724	used as a key, and the information in the main entry is assigned
725	a key of ``Data``, as outlined below.
726
727	.. tabularcolumns:: \|l\|l\|p{4in}\|
728
729	====================== ====================== ====================================================
730	key sub-key explanation
731	====================== ====================== ====================================================
732	Comments \ Text strings extracted from the original image data
733	header or a metafile. These cannot be changed by
734	the user; it may be empty.
735	Image Controls azmthOff (float) The offset to be applied to an azimuthal
736	value. Accomodates
737	detector orientations other than with the detector
738	X-axis
739	horizontal.
740	\ background image (list:str,float) The name of a tree item ("IMG ...") that is to be subtracted
741	during image integration multiplied by value. It must have the same size/shape as
742	the integrated image. NB: value < 0 for subtraction.
743	\ calibrant (str) The material used for determining the position/orientation
744	of the image. The data is obtained from :func:`ImageCalibrants`
745	and UserCalibrants.py (supplied by user).
746	\ calibdmin (float) The minimum d-spacing used during the last calibration run.
747	\ calibskip (int) The number of expected diffraction lines skipped during the last
748	calibration run.
749	\ center (list:floats) The [X,Y] point in detector coordinates (mm) where the direct beam
750	strikes the detector plane as determined by calibration. This point
751	does not have to be within the limits of the detector boundaries.
752	\ centerAzm (bool) If True then the azimuth reported for the integrated slice
753	of the image is at the center line otherwise it is at the leading edge.
754	\ color (str) The name of the colormap used to display the image. Default = 'Paired'.
755	\ cutoff (float) The minimum value of I/Ib for a point selected in a diffraction ring for
756	calibration calculations. See pixLimit for details as how point is found.
757	\ DetDepth (float) Coefficient for penetration correction to distance; accounts for diffraction
758	ring offset at higher angles. Optionally determined by calibration.
759	\ DetDepthRef (bool) If True then refine DetDepth during calibration/recalibration calculation.
760	\ distance (float) The distance (mm) from sample to detector plane.
761	\ ellipses (list:lists) Each object in ellipses is a list [center,phi,radii,color] where
762	center (list) is location (mm) of the ellipse center on the detector plane, phi is the
763	rotation of the ellipse minor axis from the x-axis, and radii are the minor & major
764	radii of the ellipse. If radii[0] is negative then parameters describe a hyperbola. Color
765	is the selected drawing color (one of 'b', 'g' ,'r') for the ellipse/hyperbola.
766	\ edgemin (float) Not used; parameter in EdgeFinder code.
767	\ fullIntegrate (bool) If True then integrate over full 360 deg azimuthal range.
768	\ GonioAngles (list:floats) The 'Omega','Chi','Phi' goniometer angles used for this image.
769	Required for texture calculations.
770	\ invert_x (bool) If True display the image with the x-axis inverted.
771	\ invert_y (bool) If True display the image with the y-axis inverted.
772	\ IOtth (list:floats) The minimum and maximum 2-theta values to be used for integration.
773	\ LRazimuth (list:floats) The minimum and maximum azimuth values to be used for integration.
774	\ Oblique (list:float,bool) If True apply a detector absorption correction using the value to the
775	intensities obtained during integration.
776	\ outAzimuths (int) The number of azimuth pie slices.
777	\ outChannels (int) The number of 2-theta steps.
778	\ pixelSize (list:ints) The X,Y dimensions (microns) of each pixel.
779	\ pixLimit (int) A box in the image with 2*pixLimit+1 edges is searched to find the maximum.
780	This value (I) along with the minimum (Ib) in the box is reported by :func:`GSASIIimage.ImageLocalMax`
781	and subject to cutoff in :func:`GSASIIimage.makeRing`.
782	Locations are used to construct rings of points for calibration calcualtions.
783	\ PolaVal (list:float,bool) If type='SASD' and if True, apply polarization correction to intensities from
784	integration using value.
785	\ rings (list:lists) Each entry is [X,Y,dsp] where X & Y are lists of x,y coordinates around a
786	diffraction ring with the same d-spacing (dsp)
787	\ ring (list) The x,y coordinates of the >5 points on an inner ring
788	selected by the user,
789	\ Range (list) The minimum & maximum values of the image
790	\ rotation (float) The angle between the x-axis and the vector about which the
791	detector is tilted. Constrained to -180 to 180 deg.
792	\ SampleShape (str) Currently only 'Cylinder'. Sample shape for Debye-Scherrer experiments; used for absorption
793	calculations.
794	\ SampleAbs (list: float,bool) Value of absorption coefficient for Debye-Scherrer experimnents, flag if True
795	to cause correction to be applied.
796	\ setDefault (bool) If True the use the image controls values for all new images to be read. (might be removed)
797	\ setRings (bool) If True then display all the selected x,y ring positions (vida supra rings) used in the calibration.
798	\ showLines (bool) If True then isplay the integration limits to be used.
799	\ size (list:int) The number of pixels on the image x & y axes
800	\ type (str) One of 'PWDR', 'SASD' or 'REFL' for powder, small angle or reflectometry data, respectively.
801	\ tilt (float) The angle the detector normal makes with the incident beam; range -90 to 90.
802	\ wavelength (float) Tha radiation wavelength (Angstroms) as entered by the user (or someday obtained from the image header).
803
804	Masks Arcs (list: lists) Each entry [2-theta,[azimuth[0],azimuth[1]],thickness] describes an arc mask
805	to be excluded from integration
806	\ Frames (list:lists) Each entry describes the x,y points (3 or more - mm) that describe a frame outside
807	of which is excluded from recalibration and integration. Only one frame is allowed.
808	\ Points (list:lists) Each entry [x,y,radius] (mm) describes an excluded spot on the image to be excluded
809	from integration.
810	\ Polygons (list:lists) Each entry is a list of 3+ [x,y] points (mm) that describe a polygon on the image
811	to be excluded from integration.
812	\ Rings (list: lists) Each entry [2-theta,thickness] describes a ring mask
813	to be excluded from integration.
814	\ Thresholds (list:[tuple,list]) [(Imin,Imax),[Imin,Imax]] This gives lower and upper limits for points on the image to be included
815	in integrsation. The tuple is the image intensity limits and the list are those set by the user.
816
817	Stress/Strain Sample phi (float) Sample rotation about vertical axis.
818	\ Sample z (float) Sample translation from the calibration sample position (for Sample phi = 0)
819	These will be restricted by space group symmetry; result of strain fit refinement.
820	\ Type (str) 'True' or 'Conventional': The strain model used for the calculation.
821	\ d-zero (list:dict) Each item is for a diffraction ring on the image; all items are from the same phase
822	and are used to determine the strain tensor.
823	The dictionary items are:
824	'Dset': (float) True d-spacing for the diffraction ring; entered by the user.
825	'Dcalc': (float) Average calculated d-spacing determined from strain coeff.
826	'Emat': (list: float) The strain tensor elements e11, e12 & e22 (e21=e12, rest are 0)
827	'Esig': (list: float) Esds for Emat from fitting.
828	'pixLimit': (int) Search range to find highest point on ring for each data point
829	'cutoff': (float) I/Ib cutoff for searching.
830	'ImxyObs': (list: lists) [[X],[Y]] observed points to be used for strain calculations.
831	'ImtaObs': (list: lists) [[d],[azm]] transformed via detector calibration from ImxyObs.
832	'ImtaCalc': (list: lists [[d],[azm]] calculated d-spacing & azimuth from fit.
833
834	====================== ====================== ====================================================
835
836	Parameter Dictionary
837	-------------------------
838
839	.. _parmDict_table:
840
841	.. index::
842	single: Parameter dictionary
843
844	The parameter dictionary contains all of the variable parameters for the refinement.
845	The dictionary keys are the name of the parameter (<phase>:<hist>:<name>:<atom>).
846	It is prepared in two ways. When loaded from the tree
847	(in :meth:`GSASII.GSASII.MakeLSParmDict` and
848	:meth:`GSASIIIO.ExportBaseclass.loadParmDict`),
849	the values are lists with two elements: ``[value, refine flag]``
850
851	When loaded from the GPX file (in
852	:func:`GSASIIstrMain.Refine` and :func:`GSASIIstrMain.SeqRefine`), the value in the
853	dict is the actual parameter value (usually a float, but sometimes a
854	letter or string flag value (such as I or A for iso/anisotropic).
855
856
857	Classes and routines
858	----------------------
859
860	'''
861	import re
862	import imp
863	import random as ran
864	import sys
865	import os.path as ospath
866	import cPickle
867	import GSASIIpath
868	import GSASIImath as G2mth
869	import GSASIIspc as G2spc
870	import numpy as np
871
872	GSASIIpath.SetVersionNumber("$Revision: 2845 $")
873
874	DefaultControls = {
875	'deriv type':'analytic Hessian',
876	'min dM/M':0.0001,'shift factor':1.,'max cyc':3,'F**2':False,
877	'UsrReject':{'minF/sig':0,'MinExt':0.01,'MaxDF/F':100.,'MaxD':500.,'MinD':0.05},
878	'Copy2Next':False,'Reverse Seq':False,'HatomFix':False,
879	'Author':'no name',
880	'FreePrm1':'Sample humidity (%)',
881	'FreePrm2':'Sample voltage (V)',
882	'FreePrm3':'Applied load (MN)',
883	'ShowCell':False,
884	}
885	'''Values to be used as defaults for the initial contents of the ``Controls``
886	data tree item.
887	'''
888	def StripUnicode(string,subs='.'):
889	'''Strip non-ASCII characters from strings
890
891	:param str string: string to strip Unicode characters from
892	:param str subs: character(s) to place into string in place of each
893	Unicode character. Defaults to '.'
894
895	:returns: a new string with only ASCII characters
896	'''
897	s = ''
898	for c in string:
899	if ord(c) < 128:
900	s += c
901	else:
902	s += subs
903	return s.encode('ascii','replace')
904
905	def MakeUniqueLabel(lbl,labellist):
906	'''Make sure that every a label is unique against a list by adding
907	digits at the end until it is not found in list.
908
909	:param str lbl: the input label
910	:param list labellist: the labels that have already been encountered
911	:returns: lbl if not found in labellist or lbl with ``_1-9`` (or
912	``_10-99``, etc.) appended at the end
913	'''
914	lbl = StripUnicode(lbl.strip(),'_')
915	if not lbl: # deal with a blank label
916	lbl = '_1'
917	if lbl not in labellist:
918	labellist.append(lbl)
919	return lbl
920	i = 1
921	prefix = lbl
922	if '_' in lbl:
923	prefix = lbl[:lbl.rfind('_')]
924	suffix = lbl[lbl.rfind('_')+1:]
925	try:
926	i = int(suffix)+1
927	except: # suffix could not be parsed
928	i = 1
929	prefix = lbl
930	while prefix+'_'+str(i) in labellist:
931	i += 1
932	else:
933	lbl = prefix+'_'+str(i)
934	labellist.append(lbl)
935	return lbl
936
937	PhaseIdLookup = {}
938	'''dict listing phase name and random Id keyed by sequential phase index as a str;
939	best to access this using :func:`LookupPhaseName`
940	'''
941	PhaseRanIdLookup = {}
942	'''dict listing phase sequential index keyed by phase random Id;
943	best to access this using :func:`LookupPhaseId`
944	'''
945	HistIdLookup = {}
946	'''dict listing histogram name and random Id, keyed by sequential histogram index as a str;
947	best to access this using :func:`LookupHistName`
948	'''
949	HistRanIdLookup = {}
950	'''dict listing histogram sequential index keyed by histogram random Id;
951	best to access this using :func:`LookupHistId`
952	'''
953	AtomIdLookup = {}
954	'''dict listing for each phase index as a str, the atom label and atom random Id,
955	keyed by atom sequential index as a str;
956	best to access this using :func:`LookupAtomLabel`
957	'''
958	AtomRanIdLookup = {}
959	'''dict listing for each phase the atom sequential index keyed by atom random Id;
960	best to access this using :func:`LookupAtomId`
961	'''
962	ShortPhaseNames = {}
963	'''a dict containing a possibly shortened and when non-unique numbered
964	version of the phase name. Keyed by the phase sequential index.
965	'''
966	ShortHistNames = {}
967	'''a dict containing a possibly shortened and when non-unique numbered
968	version of the histogram name. Keyed by the histogram sequential index.
969	'''
970
971	VarDesc = {}
972	''' This dictionary lists descriptions for GSAS-II variables,
973	as set in :func:`CompileVarDesc`. See that function for a description
974	for how keys and values are written.
975	'''
976
977	reVarDesc = {}
978	''' This dictionary lists descriptions for GSAS-II variables with
979	the same values as :attr:`VarDesc` except that keys have been compiled as
980	regular expressions. Initialized in :func:`CompileVarDesc`.
981	'''
982	# create a default space group object for P1; N.B. fails when building documentation
983	try:
984	P1SGData = G2spc.SpcGroup('P 1')[1] # data structure for default space group
985	except TypeError:
986	pass
987
988	def GetPhaseNames(fl):
989	''' Returns a list of phase names found under 'Phases' in GSASII gpx file
990	NB: there is another one of these in GSASIIstrIO.py that uses the gpx filename
991
992	:param file fl: opened .gpx file
993	:return: list of phase names
994	'''
995	PhaseNames = []
996	while True:
997	try:
998	data = cPickle.load(fl)
999	except EOFError:
1000	break
1001	datum = data[0]
1002	if 'Phases' == datum[0]:
1003	for datus in data[1:]:
1004	PhaseNames.append(datus[0])
1005	fl.seek(0) #reposition file
1006	return PhaseNames
1007
1008	def SetNewPhase(Name='New Phase',SGData=None,cell=None,Super=None):
1009	'''Create a new phase dict with default values for various parameters
1010
1011	:param str Name: Name for new Phase
1012
1013	:param dict SGData: space group data from :func:`GSASIIspc:SpcGroup`;
1014	defaults to data for P 1
1015
1016	:param list cell: unit cell parameter list; defaults to
1017	[1.0,1.0,1.0,90.,90,90.,1.]
1018
1019	'''
1020	if SGData is None: SGData = P1SGData
1021	if cell is None: cell=[1.0,1.0,1.0,90.,90,90.,1.]
1022	phaseData = {
1023	'ranId':ran.randint(0,sys.maxint),
1024	'General':{
1025	'Name':Name,
1026	'Type':'nuclear',
1027	'Modulated':False,
1028	'AtomPtrs':[3,1,7,9],
1029	'SGData':SGData,
1030	'Cell':[False,]+cell,
1031	'Pawley dmin':1.0,
1032	'Data plot type':'None',
1033	'SH Texture':{
1034	'Order':0,
1035	'Model':'cylindrical',
1036	'Sample omega':[False,0.0],
1037	'Sample chi':[False,0.0],
1038	'Sample phi':[False,0.0],
1039	'SH Coeff':[False,{}],
1040	'SHShow':False,
1041	'PFhkl':[0,0,1],
1042	'PFxyz':[0,0,1],
1043	'PlotType':'Pole figure',
1044	'Penalty':[['',],0.1,False,1.0]}},
1045	'Atoms':[],
1046	'Drawing':{},
1047	'Histograms':{},
1048	'Pawley ref':[],
1049	'RBModels':{},
1050	}
1051	if Super and Super.get('Use',False):
1052	phaseData['General'].update({'Modulated':True,'Super':True,'SuperSg':Super['ssSymb']})
1053	phaseData['General']['SSGData'] = G2spc.SSpcGroup(SGData,Super['ssSymb'])
1054	phaseData['General']['SuperVec'] = [Super['ModVec'],False,Super['maxH']]
1055
1056	return phaseData
1057
1058	def ReadCIF(URLorFile):
1059	'''Open a CIF, which may be specified as a file name or as a URL using PyCifRW
1060	(from James Hester).
1061	The open routine gets confused with DOS names that begin with a letter and colon
1062	"C:\dir\" so this routine will try to open the passed name as a file and if that
1063	fails, try it as a URL
1064
1065	:param str URLorFile: string containing a URL or a file name. Code will try first
1066	to open it as a file and then as a URL.
1067
1068	:returns: a PyCifRW CIF object.
1069	'''
1070	import CifFile as cif # PyCifRW from James Hester
1071
1072	# alternate approach:
1073	#import urllib
1074	#ciffile = 'file:'+urllib.pathname2url(filename)
1075
1076	try:
1077	fp = open(URLorFile,'r')
1078	cf = cif.ReadCif(fp)
1079	fp.close()
1080	return cf
1081	except IOError:
1082	return cif.ReadCif(URLorFile)
1083
1084	def IndexAllIds(Histograms,Phases):
1085	'''Scan through the used phases & histograms and create an index
1086	to the random numbers of phases, histograms and atoms. While doing this,
1087	confirm that assigned random numbers are unique -- just in case lightning
1088	strikes twice in the same place.
1089
1090	Note: this code assumes that the atom random Id (ranId) is the last
1091	element each atom record.
1092
1093	This is called in three places (only): :func:`GSASIIstrIO.GetUsedHistogramsAndPhases`
1094	(which loads the histograms and phases from a GPX file),
1095	:meth:`~GSASII.GSASII.GetUsedHistogramsAndPhasesfromTree`
1096	(which loads the histograms and phases from the data tree.) and
1097	:meth:`GSASIIconstrGUI.UpdateConstraints`
1098	(which displays & edits the constraints in a GUI)
1099
1100	TODO: do we need a lookup for rigid body variables?
1101	'''
1102	# process phases and atoms
1103	PhaseIdLookup.clear()
1104	PhaseRanIdLookup.clear()
1105	AtomIdLookup.clear()
1106	AtomRanIdLookup.clear()
1107	ShortPhaseNames.clear()
1108	for ph in Phases:
1109	cx,ct,cs,cia = Phases[ph]['General']['AtomPtrs']
1110	ranId = Phases[ph]['ranId']
1111	while ranId in PhaseRanIdLookup:
1112	# Found duplicate random Id! note and reassign
1113	print ("\n\n*** Phase "+str(ph)+" has repeated ranId. Fixing.\n")
1114	Phases[ph]['ranId'] = ranId = ran.randint(0,sys.maxint)
1115	pId = str(Phases[ph]['pId'])
1116	PhaseIdLookup[pId] = (ph,ranId)
1117	PhaseRanIdLookup[ranId] = pId
1118	shortname = ph #[:10]
1119	while shortname in ShortPhaseNames.values():
1120	shortname = ph[:8] + ' ('+ pId + ')'
1121	ShortPhaseNames[pId] = shortname
1122	AtomIdLookup[pId] = {}
1123	AtomRanIdLookup[pId] = {}
1124	for iatm,at in enumerate(Phases[ph]['Atoms']):
1125	ranId = at[cia+8]
1126	while ranId in AtomRanIdLookup[pId]: # check for dups
1127	print ("\n\n*** Phase "+str(ph)+" atom "+str(iatm)+" has repeated ranId. Fixing.\n")
1128	at[cia+8] = ranId = ran.randint(0,sys.maxint)
1129	AtomRanIdLookup[pId][ranId] = str(iatm)
1130	if Phases[ph]['General']['Type'] == 'macromolecular':
1131	label = '%s_%s_%s_%s'%(at[ct-1],at[ct-3],at[ct-4],at[ct-2])
1132	else:
1133	label = at[ct-1]
1134	AtomIdLookup[pId][str(iatm)] = (label,ranId)
1135	# process histograms
1136	HistIdLookup.clear()
1137	HistRanIdLookup.clear()
1138	ShortHistNames.clear()
1139	for hist in Histograms:
1140	ranId = Histograms[hist]['ranId']
1141	while ranId in HistRanIdLookup:
1142	# Found duplicate random Id! note and reassign
1143	print ("\n\n*** Histogram "+str(hist)+" has repeated ranId. Fixing.\n")
1144	Histograms[hist]['ranId'] = ranId = ran.randint(0,sys.maxint)
1145	hId = str(Histograms[hist]['hId'])
1146	HistIdLookup[hId] = (hist,ranId)
1147	HistRanIdLookup[ranId] = hId
1148	shortname = hist[:15]
1149	while shortname in ShortHistNames.values():
1150	shortname = hist[:11] + ' ('+ hId + ')'
1151	ShortHistNames[hId] = shortname
1152
1153	def LookupAtomId(pId,ranId):
1154	'''Get the atom number from a phase and atom random Id
1155
1156	:param int/str pId: the sequential number of the phase
1157	:param int ranId: the random Id assigned to an atom
1158
1159	:returns: the index number of the atom (str)
1160	'''
1161	if not AtomRanIdLookup:
1162	raise Exception,'Error: LookupAtomId called before IndexAllIds was run'
1163	if pId is None or pId == '':
1164	raise KeyError,'Error: phase is invalid (None or blank)'
1165	pId = str(pId)
1166	if pId not in AtomRanIdLookup:
1167	raise KeyError,'Error: LookupAtomId does not have phase '+pId
1168	if ranId not in AtomRanIdLookup[pId]:
1169	raise KeyError,'Error: LookupAtomId, ranId '+str(ranId)+' not in AtomRanIdLookup['+pId+']'
1170	return AtomRanIdLookup[pId][ranId]
1171
1172	def LookupAtomLabel(pId,index):
1173	'''Get the atom label from a phase and atom index number
1174
1175	:param int/str pId: the sequential number of the phase
1176	:param int index: the index of the atom in the list of atoms
1177
1178	:returns: the label for the atom (str) and the random Id of the atom (int)
1179	'''
1180	if not AtomIdLookup:
1181	raise Exception,'Error: LookupAtomLabel called before IndexAllIds was run'
1182	if pId is None or pId == '':
1183	raise KeyError,'Error: phase is invalid (None or blank)'
1184	pId = str(pId)
1185	if pId not in AtomIdLookup:
1186	raise KeyError,'Error: LookupAtomLabel does not have phase '+pId
1187	if index not in AtomIdLookup[pId]:
1188	raise KeyError,'Error: LookupAtomLabel, ranId '+str(index)+' not in AtomRanIdLookup['+pId+']'
1189	return AtomIdLookup[pId][index]
1190
1191	def LookupPhaseId(ranId):
1192	'''Get the phase number and name from a phase random Id
1193
1194	:param int ranId: the random Id assigned to a phase
1195	:returns: the sequential Id (pId) number for the phase (str)
1196	'''
1197	if not PhaseRanIdLookup:
1198	raise Exception,'Error: LookupPhaseId called before IndexAllIds was run'
1199	if ranId not in PhaseRanIdLookup:
1200	raise KeyError,'Error: LookupPhaseId does not have ranId '+str(ranId)
1201	return PhaseRanIdLookup[ranId]
1202
1203	def LookupPhaseName(pId):
1204	'''Get the phase number and name from a phase Id
1205
1206	:param int/str pId: the sequential assigned to a phase
1207	:returns: (phase,ranId) where phase is the name of the phase (str)
1208	and ranId is the random # id for the phase (int)
1209	'''
1210	if not PhaseIdLookup:
1211	raise Exception,'Error: LookupPhaseName called before IndexAllIds was run'
1212	if pId is None or pId == '':
1213	raise KeyError,'Error: phase is invalid (None or blank)'
1214	pId = str(pId)
1215	if pId not in PhaseIdLookup:
1216	raise KeyError,'Error: LookupPhaseName does not have index '+pId
1217	return PhaseIdLookup[pId]
1218
1219	def LookupHistId(ranId):
1220	'''Get the histogram number and name from a histogram random Id
1221
1222	:param int ranId: the random Id assigned to a histogram
1223	:returns: the sequential Id (hId) number for the histogram (str)
1224	'''
1225	if not HistRanIdLookup:
1226	raise Exception,'Error: LookupHistId called before IndexAllIds was run'
1227	if ranId not in HistRanIdLookup:
1228	raise KeyError,'Error: LookupHistId does not have ranId '+str(ranId)
1229	return HistRanIdLookup[ranId]
1230
1231	def LookupHistName(hId):
1232	'''Get the histogram number and name from a histogram Id
1233
1234	:param int/str hId: the sequential assigned to a histogram
1235	:returns: (hist,ranId) where hist is the name of the histogram (str)
1236	and ranId is the random # id for the histogram (int)
1237	'''
1238	if not HistIdLookup:
1239	raise Exception,'Error: LookupHistName called before IndexAllIds was run'
1240	if hId is None or hId == '':
1241	raise KeyError,'Error: histogram is invalid (None or blank)'
1242	hId = str(hId)
1243	if hId not in HistIdLookup:
1244	raise KeyError,'Error: LookupHistName does not have index '+hId
1245	return HistIdLookup[hId]
1246
1247	def fmtVarDescr(varname):
1248	'''Return a string with a more complete description for a GSAS-II variable
1249
1250	:param str varname: A full G2 variable name with 2 or 3 or 4
1251	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1252
1253	:returns: a string with the description
1254	'''
1255	s,l = VarDescr(varname)
1256	return s+": "+l
1257
1258	def VarDescr(varname):
1259	'''Return two strings with a more complete description for a GSAS-II variable
1260
1261	:param str name: A full G2 variable name with 2 or 3 or 4
1262	colons (<p>:<h>:name[:<a>] or <p>::RBname:<r>:<t>])
1263
1264	:returns: (loc,meaning) where loc describes what item the variable is mapped
1265	(phase, histogram, etc.) and meaning describes what the variable does.
1266	'''
1267
1268	# special handling for parameter names without a colons
1269	# for now, assume self-defining
1270	if varname.find(':') == -1:
1271	return "Global",varname
1272
1273	l = getVarDescr(varname)
1274	if not l:
1275	return ("invalid variable name ("+str(varname)+")!"),""
1276	# return "invalid variable name!",""
1277
1278	if not l[-1]:
1279	l[-1] = "(variable needs a definition! Set it in CompileVarDesc)"
1280
1281	if len(l) == 3: #SASD variable name!
1282	s = 'component:'+l[1]
1283	return s,l[-1]
1284	s = ""
1285	if l[0] is not None and l[1] is not None: # HAP: keep short
1286	if l[2] == "Scale": # fix up ambigous name
1287	l[5] = "Phase fraction"
1288	if l[0] == '*':
1289	lbl = 'Seq. ref.'
1290	else:
1291	lbl = ShortPhaseNames.get(l[0],'? #'+str(l[0]))
1292	if l[1] == '*':
1293	hlbl = 'Seq. ref.'
1294	else:
1295	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1296	if hlbl[:4] == 'HKLF':
1297	hlbl = 'Xtl='+hlbl[5:]
1298	elif hlbl[:4] == 'PWDR':
1299	hlbl = 'Pwd='+hlbl[5:]
1300	else:
1301	hlbl = 'Hist='+hlbl
1302	s = "Ph="+str(lbl)+" * "+str(hlbl)
1303	else:
1304	if l[2] == "Scale": # fix up ambigous name: must be scale factor, since not HAP
1305	l[5] = "Scale factor"
1306	if l[2] == 'Back': # background parameters are "special", alas
1307	s = 'Hist='+ShortHistNames.get(l[1],'? #'+str(l[1]))
1308	l[-1] += ' #'+str(l[3])
1309	elif l[4] is not None: # rigid body parameter or modulation parm
1310	lbl = ShortPhaseNames.get(l[0],'phase?')
1311	if 'RB' in l[2]: #rigid body parm
1312	s = "Res #"+str(l[3])+" body #"+str(l[4])+" in "+str(lbl)
1313	else: #modulation parm
1314	s = 'Atom %s wave %s in %s'%(LookupAtomLabel(l[0],l[3])[0],l[4],lbl)
1315	elif l[3] is not None: # atom parameter,
1316	lbl = ShortPhaseNames.get(l[0],'phase?')
1317	try:
1318	albl = LookupAtomLabel(l[0],l[3])[0]
1319	except KeyError:
1320	albl = 'Atom?'
1321	s = "Atom "+str(albl)+" in "+str(lbl)
1322	elif l[0] == '*':
1323	s = "All phases "
1324	elif l[0] is not None:
1325	lbl = ShortPhaseNames.get(l[0],'phase?')
1326	s = "Phase "+str(lbl)
1327	elif l[1] == '*':
1328	s = 'All hists'
1329	elif l[1] is not None:
1330	hlbl = ShortHistNames.get(l[1],'? #'+str(l[1]))
1331	if hlbl[:4] == 'HKLF':
1332	hlbl = 'Xtl='+hlbl[5:]
1333	elif hlbl[:4] == 'PWDR':
1334	hlbl = 'Pwd='+hlbl[5:]
1335	else:
1336	hlbl = 'Hist='+hlbl
1337	s = str(hlbl)
1338	if not s:
1339	s = 'Global'
1340	return s,l[-1]
1341
1342	def getVarDescr(varname):
1343	'''Return a short description for a GSAS-II variable
1344
1345	:param str name: A full G2 variable name with 2 or 3 or 4
1346	colons (<p>:<h>:name[:<a1>][:<a2>])
1347
1348	:returns: a six element list as [`p`,`h`,`name`,`a1`,`a2`,`description`],
1349	where `p`, `h`, `a1`, `a2` are str values or `None`, for the phase number,
1350	the histogram number and the atom number; `name` will always be
1351	a str; and `description` is str or `None`.
1352	If the variable name is incorrectly formed (for example, wrong
1353	number of colons), `None` is returned instead of a list.
1354	'''
1355	l = varname.split(':')
1356	if len(l) == 2: #SASD parameter name
1357	return varname,l[0],getDescr(l[1])
1358	if len(l) == 3:
1359	l += [None,None]
1360	elif len(l) == 4:
1361	l += [None]
1362	elif len(l) != 5:
1363	return None
1364	for i in (0,1,3,4):
1365	if l[i] == "":
1366	l[i] = None
1367	l += [getDescr(l[2])]
1368	return l
1369
1370	def CompileVarDesc():
1371	'''Set the values in the variable description lookup table (:attr:`VarDesc`)
1372	into :attr:`reVarDesc`. This is called in :func:`getDescr` so the initialization
1373	is always done before use.
1374
1375	Note that keys may contain regular expressions, where '[xyz]'
1376	matches 'x' 'y' or 'z' (equivalently '[x-z]' describes this as range of values).
1377	'.*' matches any string. For example::
1378
1379	'AUiso':'Atomic isotropic displacement parameter',
1380
1381	will match variable ``'p::AUiso:a'``.
1382	If parentheses are used in the key, the contents of those parentheses can be
1383	used in the value, such as::
1384
1385	'AU([123][123])':'Atomic anisotropic displacement parameter U\\1',
1386
1387	will match ``AU11``, ``AU23``,.. and `U11`, `U23` etc will be displayed
1388	in the value when used.
1389
1390	'''
1391	if reVarDesc: return # already done
1392	for key,value in {
1393	# derived or other sequential vars
1394	'([abc])$' : 'Lattice parameter, \\1, from Ai and Djk', # N.B. '$' prevents match if any characters follow
1395	u'\u03B1' : u'Lattice parameter, \u03B1, from Ai and Djk',
1396	u'\u03B2' : u'Lattice parameter, \u03B2, from Ai and Djk',
1397	u'\u03B3' : u'Lattice parameter, \u03B3, from Ai and Djk',
1398	# ambiguous, alas:
1399	'Scale' : 'Phase or Histogram scale factor',
1400	# Phase vars (p::<var>)
1401	'A([0-5])' : 'Reciprocal metric tensor component \\1',
1402	'[vV]ol' : 'Unit cell volume', # probably an error that both upper and lower case are used
1403	# Atom vars (p::<var>:a)
1404	'dA([xyz])$' : 'change to atomic coordinate, \\1',
1405	'A([xyz])$' : '\\1 fractional atomic coordinate',
1406	'AUiso':'Atomic isotropic displacement parameter',
1407	'AU([123][123])':'Atomic anisotropic displacement parameter U\\1',
1408	'Afrac': 'Atomic occupancy parameter',
1409	'Amul': 'Atomic site multiplicity value',
1410	'AM([xyz])$' : 'Atomic magnetic moment parameter, \\1',
1411	# Hist & Phase (HAP) vars (p:h:<var>)
1412	'Back': 'Background term',
1413	'BkPkint;(.*)':'Background peak #\\1 intensity',
1414	'BkPkpos;(.*)':'Background peak #\\1 position',
1415	'BkPksig;(.*)':'Background peak #\\1 Gaussian width',
1416	'BkPkgam;(.*)':'Background peak #\\1 Cauchy width',
1417	'Bab([AU])': 'Babinet solvent scattering coef. \\1',
1418	'D([123][123])' : 'Anisotropic strain coef. \\1',
1419	'Extinction' : 'Extinction coef.',
1420	'MD' : 'March-Dollase coef.',
1421	'Mustrain;.*' : 'Microstrain coef.',
1422	'Size;.*' : 'Crystallite size value',
1423	'eA$' : 'Cubic mustrain value',
1424	'Ep$' : 'Primary extinction',
1425	'Es$' : 'Secondary type II extinction',
1426	'Eg$' : 'Secondary type I extinction',
1427	'Flack' : 'Flack parameter',
1428	'TwinFr' : 'Twin fraction',
1429	#Histogram vars (:h:<var>)
1430	'Absorption' : 'Absorption coef.',
1431	'Displace([XY])' : 'Debye-Scherrer sample displacement \\1',
1432	'Lam' : 'Wavelength',
1433	'Polariz\.' : 'Polarization correction',
1434	'SH/L' : 'FCJ peak asymmetry correction',
1435	'([UVW])$' : 'Gaussian instrument broadening \\1',
1436	'([XY])$' : 'Cauchy instrument broadening \\1',
1437	'Zero' : 'Debye-Scherrer zero correction',
1438	'nDebye' : 'Debye model background corr. terms',
1439	'nPeaks' : 'Fixed peak background corr. terms',
1440	'RBV.*' : 'Vector rigid body parameter',
1441	'RBR.*' : 'Residue rigid body parameter',
1442	'RBRO([aijk])' : 'Residue rigid body orientation parameter',
1443	'RBRP([xyz])' : 'Residue rigid body position parameter',
1444	'RBRTr;.*' : 'Residue rigid body torsion parameter',
1445	'RBR([TLS])([123AB][123AB])' : 'Residue rigid body group disp. param.',
1446	'constr([0-9]*)' : 'Parameter from constraint',
1447	# supersymmetry parameters p::<var>:a:o 'Flen','Fcent'?
1448	'mV([0-2])$' : 'Modulation vector component \\1',
1449	'Fsin' : 'Sin site fraction modulation',
1450	'Fcos' : 'Cos site fraction modulation',
1451	'Fzero' : 'Crenel function offset', #may go away
1452	'Fwid' : 'Crenel function width',
1453	'Tmin' : 'ZigZag/Block min location',
1454	'Tmax' : 'ZigZag/Block max location',
1455	'([XYZ])max': 'ZigZag/Block max value for \\1',
1456	'([XYZ])sin' : 'Sin position wave for \\1',
1457	'([XYZ])cos' : 'Cos position wave for \\1',
1458	'U([123][123])sin$' : 'Sin thermal wave for U\\1',
1459	'U([123][123])cos$' : 'Cos thermal wave for U\\1',
1460	'M([XYZ])sin$' : 'Sin mag. moment wave for \\1',
1461	'M([XYZ])cos$' : 'Cos mag. moment wave for \\1',
1462	# PDF peak parms (l:<var>;l = peak no.)
1463	'PDFpos' : 'PDF peak position',
1464	'PDFmag' : 'PDF peak magnitude',
1465	'PDFsig' : 'PDF peak std. dev.',
1466	# SASD vars (l:<var>;l = component)
1467	'Aspect ratio' : 'Particle aspect ratio',
1468	'Length' : 'Cylinder length',
1469	'Diameter' : 'Cylinder/disk diameter',
1470	'Thickness' : 'Disk thickness',
1471	'Shell thickness' : 'Multiplier to get inner(<1) or outer(>1) sphere radius',
1472	'Dist' : 'Interparticle distance',
1473	'VolFr' : 'Dense scatterer volume fraction',
1474	'epis' : 'Sticky sphere epsilon',
1475	'Sticky' : 'Stickyness',
1476	'Depth' : 'Well depth',
1477	'Width' : 'Well width',
1478	'Volume' : 'Particle volume',
1479	'Radius' : 'Sphere/cylinder/disk radius',
1480	'Mean' : 'Particle mean radius',
1481	'StdDev' : 'Standard deviation in Mean',
1482	'G$': 'Guinier prefactor',
1483	'Rg$': 'Guinier radius of gyration',
1484	'B$': 'Porod prefactor',
1485	'P$': 'Porod power',
1486	'Cutoff': 'Porod cutoff',
1487	'PkInt': 'Bragg peak intensity',
1488	'PkPos': 'Bragg peak position',
1489	'PkSig': 'Bragg peak sigma',
1490	'PkGam': 'Bragg peak gamma',
1491	'e([12][12])' : 'strain tensor e\1', # strain vars e11, e22, e12
1492	'Dcalc': 'Calc. d-spacing',
1493	'Back$': 'background parameter',
1494	'pos$': 'peak position',
1495	'int$': 'peak intensity',
1496	'WgtFrac':'phase weight fraction',
1497	}.items():
1498	VarDesc[key] = value
1499	reVarDesc[re.compile(key)] = value
1500
1501	def getDescr(name):
1502	'''Return a short description for a GSAS-II variable
1503
1504	:param str name: The descriptive part of the variable name without colons (:)
1505
1506	:returns: a short description or None if not found
1507	'''
1508
1509	CompileVarDesc() # compile the regular expressions, if needed
1510	for key in reVarDesc:
1511	m = key.match(name)
1512	if m:
1513	reVarDesc[key]
1514	return m.expand(reVarDesc[key])
1515	return None
1516
1517	def GenWildCard(varlist):
1518	'''Generate wildcard versions of G2 variables. These introduce '*'
1519	for a phase, histogram or atom number (but only for one of these
1520	fields) but only when there is more than one matching variable in the
1521	input variable list. So if the input is this::
1522
1523	varlist = ['0::AUiso:0', '0::AUiso:1', '1::AUiso:0']
1524
1525	then the output will be this::
1526
1527	wildList = ['::AUiso:0', '0::AUiso:']
1528
1529	:param list varlist: an input list of GSAS-II variable names
1530	(such as 0::AUiso:0)
1531
1532	:returns: wildList, the generated list of wild card variable names.
1533	'''
1534	wild = []
1535	for i in (0,1,3):
1536	currentL = varlist[:]
1537	while currentL:
1538	item1 = currentL.pop(0)
1539	i1splt = item1.split(':')
1540	if i >= len(i1splt): continue
1541	if i1splt[i]:
1542	nextL = []
1543	i1splt[i] = '[0-9]+'
1544	rexp = re.compile(':'.join(i1splt))
1545	matchlist = [item1]
1546	for nxtitem in currentL:
1547	if rexp.match(nxtitem):
1548	matchlist += [nxtitem]
1549	else:
1550	nextL.append(nxtitem)
1551	if len(matchlist) > 1:
1552	i1splt[i] = '*'
1553	wild.append(':'.join(i1splt))
1554	currentL = nextL
1555	return wild
1556
1557	def LookupWildCard(varname,varlist):
1558	'''returns a list of variable names from list varname
1559	that match wildcard name in varname
1560
1561	:param str varname: a G2 variable name containing a wildcard
1562	(such as \*::var)
1563	:param list varlist: the list of all variable names used in
1564	the current project
1565	:returns: a list of matching GSAS-II variables (may be empty)
1566	'''
1567	rexp = re.compile(varname.replace('*','[0-9]+'))
1568	return sorted([var for var in varlist if rexp.match(var)])
1569
1570
1571	def _lookup(dic,key):
1572	'''Lookup a key in a dictionary, where None returns an empty string
1573	but an unmatched key returns a question mark. Used in :class:`G2VarObj`
1574	'''
1575	if key is None:
1576	return ""
1577	elif key == "*":
1578	return "*"
1579	else:
1580	return dic.get(key,'?')
1581
1582	class G2VarObj(object):
1583	'''Defines a GSAS-II variable either using the phase/atom/histogram
1584	unique Id numbers or using a character string that specifies
1585	variables by phase/atom/histogram number (which can change).
1586	Note that :func:`LoadID` should be used to (re)load the current Ids
1587	before creating or later using the G2VarObj object.
1588
1589	This can store rigid body variables, but does not translate the residue # and
1590	body # to/from random Ids
1591
1592	A :class:`G2VarObj` object can be created with a single parameter:
1593
1594	:param str/tuple varname: a single value can be used to create a :class:`G2VarObj`
1595	object. If a string, it must be of form "p:h:var" or "p:h:var:a", where
1596
1597	* p is the phase number (which may be left blank or may be '*' to indicate all phases);
1598	* h is the histogram number (which may be left blank or may be '*' to indicate all histograms);
1599	* a is the atom number (which may be left blank in which case the third colon is omitted).
1600	The atom number can be specified as '' if a phase number is specified (not as '').
1601	For rigid body variables, specify a will be a string of form "residue:body#"
1602
1603	Alternately a single tuple of form (Phase,Histogram,VarName,AtomID) can be used, where
1604	Phase, Histogram, and AtomID are None or are ranId values (or one can be '*')
1605	and VarName is a string. Note that if Phase is '*' then the AtomID is an atom number.
1606	For a rigid body variables, AtomID is a string of form "residue:body#".
1607
1608	If four positional arguments are supplied, they are:
1609
1610	:param str/int phasenum: The number for the phase (or None or '*')
1611	:param str/int histnum: The number for the histogram (or None or '*')
1612	:param str varname: a single value can be used to create a :class:`G2VarObj`
1613	:param str/int atomnum: The number for the atom (or None or '*')
1614
1615	'''
1616	IDdict = {}
1617	IDdict['phases'] = {}
1618	IDdict['hists'] = {}
1619	IDdict['atoms'] = {}
1620	def __init__(self,*args):
1621	self.phase = None
1622	self.histogram = None
1623	self.name = ''
1624	self.atom = None
1625	if len(args) == 1 and (type(args[0]) is list or type(args[0]) is tuple) and len(args[0]) == 4:
1626	# single arg with 4 values
1627	self.phase,self.histogram,self.name,self.atom = args[0]
1628	elif len(args) == 1 and ':' in args[0]:
1629	#parse a string
1630	lst = args[0].split(':')
1631	if lst[0] == '*':
1632	self.phase = '*'
1633	if len(lst) > 3:
1634	self.atom = lst[3]
1635	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
1636	elif lst[1] == '*':
1637	self.histogram = '*'
1638	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
1639	else:
1640	self.histogram = HistIdLookup.get(lst[1],[None,None])[1]
1641	self.phase = PhaseIdLookup.get(lst[0],[None,None])[1]
1642	if len(lst) == 4:
1643	if lst[3] == '*':
1644	self.atom = '*'
1645	else:
1646	self.atom = AtomIdLookup[lst[0]].get(lst[3],[None,None])[1]
1647	elif len(lst) == 5:
1648	self.atom = lst[3]+":"+lst[4]
1649	elif len(lst) == 3:
1650	pass
1651	else:
1652	raise Exception,"Too many colons in var name "+str(args[0])
1653	self.name = lst[2]
1654	elif len(args) == 4:
1655	if args[0] == '*':
1656	self.phase = '*'
1657	self.atom = args[3]
1658	else:
1659	self.phase = PhaseIdLookup.get(str(args[0]),[None,None])[1]
1660	if args[3] == '*':
1661	self.atom = '*'
1662	elif args[0] is not None:
1663	self.atom = AtomIdLookup[args[0]].get(str(args[3]),[None,None])[1]
1664	if args[1] == '*':
1665	self.histogram = '*'
1666	else:
1667	self.histogram = HistIdLookup.get(str(args[1]),[None,None])[1]
1668	self.name = args[2]
1669	else:
1670	raise Exception,"Incorrectly called GSAS-II parameter name"
1671
1672	#print "DEBUG: created ",self.phase,self.histogram,self.name,self.atom
1673
1674	def __str__(self):
1675	return self.varname()
1676
1677	def varname(self):
1678	'''Formats the GSAS-II variable name as a "traditional" GSAS-II variable
1679	string (p:h:<var>:a) or (p:h:<var>)
1680
1681	:returns: the variable name as a str
1682	'''
1683	a = ""
1684	if self.phase == "*":
1685	ph = "*"
1686	if self.atom:
1687	a = ":" + str(self.atom)
1688	else:
1689	ph = _lookup(PhaseRanIdLookup,self.phase)
1690	if self.atom == '*':
1691	a = ':*'
1692	elif self.atom:
1693	if ":" in str(self.atom):
1694	a = ":" + str(self.atom)
1695	elif ph in AtomRanIdLookup:
1696	a = ":" + AtomRanIdLookup[ph].get(self.atom,'?')
1697	else:
1698	a = ":?"
1699	if self.histogram == "*":
1700	hist = "*"
1701	else:
1702	hist = _lookup(HistRanIdLookup,self.histogram)
1703	s = (ph + ":" + hist + ":" + str(self.name)) + a
1704	return s
1705
1706	def __repr__(self):
1707	'''Return the detailed contents of the object
1708	'''
1709	s = "<"
1710	if self.phase == '*':
1711	s += "Phases: all; "
1712	if self.atom is not None:
1713	if ":" in str(self.atom):
1714	s += "Rigid body" + str(self.atom) + "; "
1715	else:
1716	s += "Atom #" + str(self.atom) + "; "
1717	elif self.phase is not None:
1718	ph = _lookup(PhaseRanIdLookup,self.phase)
1719	s += "Phase: rId=" + str(self.phase) + " (#"+ ph + "); "
1720	if self.atom == '*':
1721	s += "Atoms: all; "
1722	elif ":" in str(self.atom):
1723	s += "Rigid body" + str(self.atom) + "; "
1724	elif self.atom is not None:
1725	s += "Atom rId=" + str(self.atom)
1726	if ph in AtomRanIdLookup:
1727	s += " (#" + AtomRanIdLookup[ph].get(self.atom,'?') + "); "
1728	else:
1729	s += " (#? -- not found!); "
1730	if self.histogram == '*':
1731	s += "Histograms: all; "
1732	elif self.histogram is not None:
1733	hist = _lookup(HistRanIdLookup,self.histogram)
1734	s += "Histogram: rId=" + str(self.histogram) + " (#"+ hist + "); "
1735	s += 'Variable name="' + str(self.name) + '">'
1736	return s+" ("+self.varname()+")"
1737
1738	def __eq__(self, other):
1739	if type(other) is type(self):
1740	return (self.phase == other.phase and
1741	self.histogram == other.histogram and
1742	self.name == other.name and
1743	self.atom == other.atom)
1744	return False
1745
1746	def _show(self):
1747	'For testing, shows the current lookup table'
1748	print 'phases', self.IDdict['phases']
1749	print 'hists', self.IDdict['hists']
1750	print 'atomDict', self.IDdict['atoms']
1751
1752	#==========================================================================
1753	def SetDefaultSample():
1754	'Fills in default items for the Sample dictionary for Debye-Scherrer & SASD'
1755	return {
1756	'InstrName':'',
1757	'ranId':ran.randint(0,sys.maxint),
1758	'Scale':[1.0,True],'Type':'Debye-Scherrer','Absorption':[0.0,False],
1759	'DisplaceX':[0.0,False],'DisplaceY':[0.0,False],'Diffuse':[],
1760	'Temperature':300.,'Pressure':0.1,'Time':0.0,
1761	'FreePrm1':0.,'FreePrm2':0.,'FreePrm3':0.,
1762	'Gonio. radius':200.0,
1763	'Omega':0.0,'Chi':0.0,'Phi':0.0,'Azimuth':0.0,
1764	#SASD items
1765	'Materials':[{'Name':'vacuum','VolFrac':1.0,},{'Name':'vacuum','VolFrac':0.0,}],
1766	'Thick':1.0,'Contrast':[0.0,0.0], #contrast & anomalous contrast
1767	'Trans':1.0, #measured transmission
1768	'SlitLen':0.0, #Slit length - in Q(A-1)
1769	}
1770	######################################################################
1771	class ImportBaseclass(object):
1772	'''Defines a base class for the reading of input files (diffraction
1773	data, coordinates,...). See :ref:`Writing a Import Routine<Import_routines>`
1774	for an explanation on how to use a subclass of this class.
1775	'''
1776	class ImportException(Exception):
1777	'''Defines an Exception that is used when an import routine hits an expected error,
1778	usually in .Reader.
1779
1780	Good practice is that the Reader should define a value in self.errors that
1781	tells the user some information about what is wrong with their file.
1782	'''
1783	pass
1784
1785	UseReader = True # in __init__ set value of self.UseReader to False to skip use of current importer
1786	def __init__(self,formatName,longFormatName=None,
1787	extensionlist=[],strictExtension=False,):
1788	self.formatName = formatName # short string naming file type
1789	if longFormatName: # longer string naming file type
1790	self.longFormatName = longFormatName
1791	else:
1792	self.longFormatName = formatName
1793	# define extensions that are allowed for the file type
1794	# for windows, remove any extensions that are duplicate, as case is ignored
1795	if sys.platform == 'windows' and extensionlist:
1796	extensionlist = list(set([s.lower() for s in extensionlist]))
1797	self.extensionlist = extensionlist
1798	# If strictExtension is True, the file will not be read, unless
1799	# the extension matches one in the extensionlist
1800	self.strictExtension = strictExtension
1801	self.errors = ''
1802	self.warnings = ''
1803	self.SciPy = False #image reader needed scipy
1804	# used for readers that will use multiple passes to read
1805	# more than one data block
1806	self.repeat = False
1807	self.selections = []
1808	self.repeatcount = 0
1809	self.readfilename = '?'
1810	self.scriptable = False
1811	#print 'created',self.__class__
1812
1813	def ReInitialize(self):
1814	'Reinitialize the Reader to initial settings'
1815	self.errors = ''
1816	self.warnings = ''
1817	self.SciPy = False #image reader needed scipy
1818	self.repeat = False
1819	self.repeatcount = 0
1820	self.readfilename = '?'
1821
1822
1823	# def Reader(self, filename, filepointer, ParentFrame=None, **unused):
1824	# '''This method must be supplied in the child class to read the file.
1825	# if the read fails either return False or raise an Exception
1826	# preferably of type ImportException.
1827	# '''
1828	# #start reading
1829	# raise ImportException("Error occurred while...")
1830	# self.errors += "Hint for user on why the error occur
1831	# return False # if an error occurs
1832	# return True # if read OK
1833
1834	def ExtensionValidator(self, filename):
1835	'''This methods checks if the file has the correct extension
1836	Return False if this filename will not be supported by this reader
1837	Return True if the extension matches the list supplied by the reader
1838	Return None if the reader allows un-registered extensions
1839	'''
1840	if filename:
1841	ext = ospath.splitext(filename)[1]
1842	if sys.platform == 'windows': ext = ext.lower()
1843	if ext in self.extensionlist: return True
1844	if self.strictExtension: return False
1845	return None
1846
1847	def ContentsValidator(self, filepointer):
1848	'''This routine will attempt to determine if the file can be read
1849	with the current format.
1850	This will typically be overridden with a method that
1851	takes a quick scan of [some of]
1852	the file contents to do a "sanity" check if the file
1853	appears to match the selected format.
1854	Expected to be called via self.Validator()
1855	'''
1856	#filepointer.seek(0) # rewind the file pointer
1857	return True
1858
1859	def CIFValidator(self, filepointer):
1860	'''A :meth:`ContentsValidator` for use to validate CIF files.
1861	'''
1862	for i,l in enumerate(filepointer):
1863	if i >= 1000: return True
1864	'''Encountered only blank lines or comments in first 1000
1865	lines. This is unlikely, but assume it is CIF anyway, since we are
1866	even less likely to find a file with nothing but hashes and
1867	blank lines'''
1868	line = l.strip()
1869	if len(line) == 0: # ignore blank lines
1870	continue
1871	elif line.startswith('#'): # ignore comments
1872	continue
1873	elif line.startswith('data_'): # on the right track, accept this file
1874	return True
1875	else: # found something invalid
1876	self.errors = 'line '+str(i+1)+' contains unexpected data:\n'
1877	if all([ord(c) < 128 and ord(c) != 0 for c in str(l)]): # show only if ASCII
1878	self.errors += ' '+str(l)
1879	else:
1880	self.errors += ' (binary)'
1881	self.errors += '\n Note: a CIF should only have blank lines or comments before'
1882	self.errors += '\n a data_ statement begins a block.'
1883	return False
1884
1885	######################################################################
1886	class ImportPhase(ImportBaseclass):
1887	'''Defines a base class for the reading of files with coordinates
1888
1889	Objects constructed that subclass this (in import/G2phase_*.py etc.) will be used
1890	in :meth:`GSASII.GSASII.OnImportPhase`.
1891	See :ref:`Writing a Import Routine<Import_Routines>`
1892	for an explanation on how to use this class.
1893
1894	'''
1895	def __init__(self,formatName,longFormatName=None,extensionlist=[],
1896	strictExtension=False,):
1897	# call parent __init__
1898	ImportBaseclass.__init__(self,formatName,longFormatName,
1899	extensionlist,strictExtension)
1900	self.Phase = None # a phase must be created with G2IO.SetNewPhase in the Reader
1901	self.Constraints = None
1902
1903	######################################################################
1904	class ImportStructFactor(ImportBaseclass):
1905	'''Defines a base class for the reading of files with tables
1906	of structure factors.
1907
1908	Structure factors are read with a call to :meth:`GSASII.GSASII.OnImportSfact`
1909	which in turn calls :meth:`GSASII.GSASII.OnImportGeneric`, which calls
1910	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
1911	:meth:`Reader`.
1912
1913	See :ref:`Writing a Import Routine<Import_Routines>`
1914	for an explanation on how to use import classes in general. The specifics
1915	for reading a structure factor histogram require that
1916	the ``Reader()`` routine in the import
1917	class need to do only a few things: It
1918	should load :attr:`RefDict` item ``'RefList'`` with the reflection list,
1919	and set :attr:`Parameters` with the instrument parameters
1920	(initialized with :meth:`InitParameters` and set with :meth:`UpdateParameters`).
1921	'''
1922	def __init__(self,formatName,longFormatName=None,extensionlist=[],
1923	strictExtension=False,):
1924	ImportBaseclass.__init__(self,formatName,longFormatName,
1925	extensionlist,strictExtension)
1926
1927	# define contents of Structure Factor entry
1928	self.Parameters = []
1929	'self.Parameters is a list with two dicts for data parameter settings'
1930	self.InitParameters()
1931	self.RefDict = {'RefList':[],'FF':{},'Super':0}
1932	self.Banks = [] #for multi bank data (usually TOF)
1933	'''self.RefDict is a dict containing the reflection information, as read from the file.
1934	Item 'RefList' contains the reflection information. See the
1935	:ref:`Single Crystal Reflection Data Structure<XtalRefl_table>`
1936	for the contents of each row. Dict element 'FF'
1937	contains the form factor values for each element type; if this entry
1938	is left as initialized (an empty list) it will be initialized as needed later.
1939	'''
1940	def ReInitialize(self):
1941	'Reinitialize the Reader to initial settings'
1942	ImportBaseclass.ReInitialize(self)
1943	self.InitParameters()
1944	self.Banks = [] #for multi bank data (usually TOF)
1945	self.RefDict = {'RefList':[],'FF':{},'Super':0}
1946
1947	def InitParameters(self):
1948	'initialize the instrument parameters structure'
1949	Lambda = 0.70926
1950	HistType = 'SXC'
1951	self.Parameters = [{'Type':[HistType,HistType], # create the structure
1952	'Lam':[Lambda,Lambda]
1953	}, {}]
1954	'Parameters is a list with two dicts for data parameter settings'
1955
1956	def UpdateParameters(self,Type=None,Wave=None):
1957	'Revise the instrument parameters'
1958	if Type is not None:
1959	self.Parameters[0]['Type'] = [Type,Type]
1960	if Wave is not None:
1961	self.Parameters[0]['Lam'] = [Wave,Wave]
1962
1963	######################################################################
1964	class ImportPowderData(ImportBaseclass):
1965	'''Defines a base class for the reading of files with powder data.
1966
1967	Objects constructed that subclass this (in import/G2pwd_*.py etc.) will be used
1968	in :meth:`GSASII.GSASII.OnImportPowder`.
1969	See :ref:`Writing a Import Routine<Import_Routines>`
1970	for an explanation on how to use this class.
1971	'''
1972	def __init__(self,formatName,longFormatName=None,
1973	extensionlist=[],strictExtension=False,):
1974	ImportBaseclass.__init__(self,formatName,longFormatName,
1975	extensionlist,strictExtension)
1976	self.clockWd = None # used in TOF
1977	self.ReInitialize()
1978
1979	def ReInitialize(self):
1980	'Reinitialize the Reader to initial settings'
1981	ImportBaseclass.ReInitialize(self)
1982	self.powderentry = ['',None,None] # (filename,Pos,Bank)
1983	self.powderdata = [] # Powder dataset
1984	'''A powder data set is a list with items [x,y,w,yc,yb,yd]:
1985	np.array(x), # x-axis values
1986	np.array(y), # powder pattern intensities
1987	np.array(w), # 1/sig(intensity)^2 values (weights)
1988	np.array(yc), # calc. intensities (zero)
1989	np.array(yb), # calc. background (zero)
1990	np.array(yd), # obs-calc profiles
1991	'''
1992	self.comments = []
1993	self.idstring = ''
1994	self.Sample = SetDefaultSample() # default sample parameters
1995	self.Controls = {} # items to be placed in top-level Controls
1996	self.GSAS = None # used in TOF
1997	self.repeat_instparm = True # Should a parm file be
1998	# used for multiple histograms?
1999	self.instparm = None # name hint from file of instparm to use
2000	self.instfile = '' # full path name to instrument parameter file
2001	self.instbank = '' # inst parm bank number
2002	self.instmsg = '' # a label that gets printed to show
2003	# where instrument parameters are from
2004	self.numbanks = 1
2005	self.instdict = {} # place items here that will be transferred to the instrument parameters
2006	self.pwdparms = {} # place parameters that are transferred directly to the tree
2007	# here (typically from an existing GPX file)
2008	######################################################################
2009	class ImportSmallAngleData(ImportBaseclass):
2010	'''Defines a base class for the reading of files with small angle data.
2011	See :ref:`Writing a Import Routine<Import_Routines>`
2012	for an explanation on how to use this class.
2013	'''
2014	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2015	strictExtension=False,):
2016
2017	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2018	strictExtension)
2019	self.ReInitialize()
2020
2021	def ReInitialize(self):
2022	'Reinitialize the Reader to initial settings'
2023	ImportBaseclass.ReInitialize(self)
2024	self.smallangleentry = ['',None,None] # (filename,Pos,Bank)
2025	self.smallangledata = [] # SASD dataset
2026	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2027	np.array(x), # x-axis values
2028	np.array(y), # powder pattern intensities
2029	np.array(w), # 1/sig(intensity)^2 values (weights)
2030	np.array(yc), # calc. intensities (zero)
2031	np.array(yd), # obs-calc profiles
2032	np.array(yb), # preset bkg
2033	'''
2034	self.comments = []
2035	self.idstring = ''
2036	self.Sample = SetDefaultSample()
2037	self.GSAS = None # used in TOF
2038	self.clockWd = None # used in TOF
2039	self.numbanks = 1
2040	self.instdict = {} # place items here that will be transferred to the instrument parameters
2041
2042	######################################################################
2043	class ImportReflectometryData(ImportBaseclass):
2044	'''Defines a base class for the reading of files with reflectometry data.
2045	See :ref:`Writing a Import Routine<Import_Routines>`
2046	for an explanation on how to use this class.
2047	'''
2048	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2049	strictExtension=False,):
2050
2051	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2052	strictExtension)
2053	self.ReInitialize()
2054
2055	def ReInitialize(self):
2056	'Reinitialize the Reader to initial settings'
2057	ImportBaseclass.ReInitialize(self)
2058	self.reflectometryentry = ['',None,None] # (filename,Pos,Bank)
2059	self.reflectometrydata = [] # SASD dataset
2060	'''A small angle data set is a list with items [x,y,w,yc,yd]:
2061	np.array(x), # x-axis values
2062	np.array(y), # powder pattern intensities
2063	np.array(w), # 1/sig(intensity)^2 values (weights)
2064	np.array(yc), # calc. intensities (zero)
2065	np.array(yd), # obs-calc profiles
2066	np.array(yb), # preset bkg
2067	'''
2068	self.comments = []
2069	self.idstring = ''
2070	self.Sample = SetDefaultSample()
2071	self.GSAS = None # used in TOF
2072	self.clockWd = None # used in TOF
2073	self.numbanks = 1
2074	self.instdict = {} # place items here that will be transferred to the instrument parameters
2075
2076	######################################################################
2077	class ImportPDFData(ImportBaseclass):
2078	'''Defines a base class for the reading of files with PDF G(R) data.
2079	See :ref:`Writing a Import Routine<Import_Routines>`
2080	for an explanation on how to use this class.
2081	'''
2082	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2083	strictExtension=False,):
2084
2085	ImportBaseclass.__init__(self,formatName,longFormatName,extensionlist,
2086	strictExtension)
2087	self.ReInitialize()
2088
2089	def ReInitialize(self):
2090	'Reinitialize the Reader to initial settings'
2091	ImportBaseclass.ReInitialize(self)
2092	self.pdfentry = ['',None,None] # (filename,Pos,Bank)
2093	self.pdfdata = [] # PDF G(R) dataset
2094	'''A pdf g(r) data set is a list with items [x,y]:
2095	np.array(x), # r-axis values
2096	np.array(y), # pdf g(r)
2097	'''
2098	self.comments = []
2099	self.idstring = ''
2100	self.numbanks = 1
2101
2102	######################################################################
2103	class ImportImage(ImportBaseclass):
2104	'''Defines a base class for the reading of images
2105
2106	Images are read in only these places:
2107
2108	* Initial reading is typically done from a menu item
2109	with a call to :meth:`GSASII.GSASII.OnImportImage`
2110	which in turn calls :meth:`GSASII.GSASII.OnImportGeneric`. That calls
2111	methods :meth:`ExtensionValidator`, :meth:`ContentsValidator` and
2112	:meth:`Reader`. This returns a list of reader objects for each read image.
2113
2114	* Images are read alternatively in :func:`GSASIIIO.ReadImages`, which puts image info
2115	directly into the data tree.
2116
2117	* Images are reloaded with :func:`GSASIIIO.GetImageData`.
2118
2119	.. _Image_import_routines:
2120
2121	When reading an image, the ``Reader()`` routine in the ImportImage class
2122	should set:
2123
2124	* :attr:`Comments`: a list of strings (str),
2125	* :attr:`Npix`: the number of pixels in the image (int),
2126	* :attr:`Image`: the actual image as a numpy array (np.array)
2127	* :attr:`Data`: a dict defining image parameters (dict). Within this dict the following
2128	data items are needed:
2129
2130	* 'pixelSize': size of each pixel in microns (such as ``[200,200]``.
2131	* 'wavelength': wavelength in Angstoms.
2132	* 'distance': distance of detector from sample in cm.
2133	* 'center': uncalibrated center of beam on detector (such as ``[204.8,204.8]``.
2134	* 'size': size of image (such as ``[2048,2048]``).
2135	* 'ImageTag': image number or other keyword used to retrieve image from
2136	a multi-image data file (defaults to ``1`` if not specified).
2137	* 'sumfile': holds sum image file name if a sum was produced from a multi image file
2138
2139	optional data items:
2140
2141	* :attr:`repeat`: set to True if there are additional images to
2142	read in the file, False otherwise
2143	* :attr:`repeatcount`: set to the number of the image.
2144
2145	Note that the above is initialized with :meth:`InitParameters`.
2146	(Also see :ref:`Writing a Import Routine<Import_Routines>`
2147	for an explanation on how to use import classes in general.)
2148	'''
2149	def __init__(self,formatName,longFormatName=None,extensionlist=[],
2150	strictExtension=False,):
2151	ImportBaseclass.__init__(self,formatName,longFormatName,
2152	extensionlist,strictExtension)
2153	self.InitParameters()
2154
2155	def ReInitialize(self):
2156	'Reinitialize the Reader to initial settings -- not used at present'
2157	ImportBaseclass.ReInitialize(self)
2158	self.InitParameters()
2159
2160	def InitParameters(self):
2161	'initialize the instrument parameters structure'
2162	self.Comments = ['No comments']
2163	self.Data = {}
2164	self.Npix = 0
2165	self.Image = None
2166	self.repeat = False
2167	self.repeatcount = 1
2168	self.sumfile = ''
2169
2170	def LoadImage(self,ParentFrame,imagefile,imagetag=None):
2171	'''Optionally, call this after reading in an image to load it into the tree.
2172	This saves time by preventing a reread of the same information.
2173	'''
2174	if ParentFrame:
2175	ParentFrame.ImageZ = self.Image # store the image for plotting
2176	ParentFrame.oldImagefile = imagefile # save the name of the last image file read
2177	ParentFrame.oldImageTag = imagetag # save the tag of the last image file read
2178
2179	#################################################################################################
2180	# shortcut routines
2181	exp = np.exp
2182	sind = sin = s = lambda x: np.sin(x*np.pi/180.)
2183	cosd = cos = c = lambda x: np.cos(x*np.pi/180.)
2184	tand = tan = t = lambda x: np.tan(x*np.pi/180.)
2185	sqrt = sq = lambda x: np.sqrt(x)
2186	pi = lambda: np.pi
2187	class ExpressionObj(object):
2188	'''Defines an object with a user-defined expression, to be used for
2189	secondary fits or restraints. Object is created null, but is changed
2190	using :meth:`LoadExpression`. This contains only the minimum
2191	information that needs to be stored to save and load the expression
2192	and how it is mapped to GSAS-II variables.
2193	'''
2194	def __init__(self):
2195	self.expression = ''
2196	'The expression as a text string'
2197	self.assgnVars = {}
2198	'''A dict where keys are label names in the expression mapping to a GSAS-II
2199	variable. The value a G2 variable name.
2200	Note that the G2 variable name may contain a wild-card and correspond to
2201	multiple values.
2202	'''
2203	self.freeVars = {}
2204	'''A dict where keys are label names in the expression mapping to a free
2205	parameter. The value is a list with:
2206
2207	* a name assigned to the parameter
2208	* a value for to the parameter and
2209	* a flag to determine if the variable is refined.
2210	'''
2211	self.depVar = None
2212
2213	self.lastError = ('','')
2214	'''Shows last encountered error in processing expression
2215	(list of 1-3 str values)'''
2216
2217	self.distance_dict = None # to be used for defining atom phase/symmetry info
2218	self.distance_atoms = None # to be used for defining atom distances
2219
2220	def LoadExpression(self,expr,exprVarLst,varSelect,varName,varValue,varRefflag):
2221	'''Load the expression and associated settings into the object. Raises
2222	an exception if the expression is not parsed, if not all functions
2223	are defined or if not all needed parameter labels in the expression
2224	are defined.
2225
2226	This will not test if the variable referenced in these definitions
2227	are actually in the parameter dictionary. This is checked when the
2228	computation for the expression is done in :meth:`SetupCalc`.
2229
2230	:param str expr: the expression
2231	:param list exprVarLst: parameter labels found in the expression
2232	:param dict varSelect: this will be 0 for Free parameters
2233	and non-zero for expression labels linked to G2 variables.
2234	:param dict varName: Defines a name (str) associated with each free parameter
2235	:param dict varValue: Defines a value (float) associated with each free parameter
2236	:param dict varRefflag: Defines a refinement flag (bool)
2237	associated with each free parameter
2238	'''
2239	self.expression = expr
2240	self.compiledExpr = None
2241	self.freeVars = {}
2242	self.assgnVars = {}
2243	for v in exprVarLst:
2244	if varSelect[v] == 0:
2245	self.freeVars[v] = [
2246	varName.get(v),
2247	varValue.get(v),
2248	varRefflag.get(v),
2249	]
2250	else:
2251	self.assgnVars[v] = varName[v]
2252	self.CheckVars()
2253
2254	def EditExpression(self,exprVarLst,varSelect,varName,varValue,varRefflag):
2255	'''Load the expression and associated settings from the object into
2256	arrays used for editing.
2257
2258	:param list exprVarLst: parameter labels found in the expression
2259	:param dict varSelect: this will be 0 for Free parameters
2260	and non-zero for expression labels linked to G2 variables.
2261	:param dict varName: Defines a name (str) associated with each free parameter
2262	:param dict varValue: Defines a value (float) associated with each free parameter
2263	:param dict varRefflag: Defines a refinement flag (bool)
2264	associated with each free parameter
2265
2266	:returns: the expression as a str
2267	'''
2268	for v in self.freeVars:
2269	varSelect[v] = 0
2270	varName[v] = self.freeVars[v][0]
2271	varValue[v] = self.freeVars[v][1]
2272	varRefflag[v] = self.freeVars[v][2]
2273	for v in self.assgnVars:
2274	varSelect[v] = 1
2275	varName[v] = self.assgnVars[v]
2276	return self.expression
2277
2278	def GetVaried(self):
2279	'Returns the names of the free parameters that will be refined'
2280	return ["::"+self.freeVars[v][0] for v in self.freeVars if self.freeVars[v][2]]
2281
2282	def GetVariedVarVal(self):
2283	'Returns the names and values of the free parameters that will be refined'
2284	return [("::"+self.freeVars[v][0],self.freeVars[v][1]) for v in self.freeVars if self.freeVars[v][2]]
2285
2286	def UpdateVariedVars(self,varyList,values):
2287	'Updates values for the free parameters (after a refinement); only updates refined vars'
2288	for v in self.freeVars:
2289	if not self.freeVars[v][2]: continue
2290	if "::"+self.freeVars[v][0] not in varyList: continue
2291	indx = varyList.index("::"+self.freeVars[v][0])
2292	self.freeVars[v][1] = values[indx]
2293
2294	def GetIndependentVars(self):
2295	'Returns the names of the required independent parameters used in expression'
2296	return [self.assgnVars[v] for v in self.assgnVars]
2297
2298	def CheckVars(self):
2299	'''Check that the expression can be parsed, all functions are
2300	defined and that input loaded into the object is internally
2301	consistent. If not an Exception is raised.
2302
2303	:returns: a dict with references to packages needed to
2304	find functions referenced in the expression.
2305	'''
2306	ret = self.ParseExpression(self.expression)
2307	if not ret:
2308	raise Exception("Expression parse error")
2309	exprLblList,fxnpkgdict = ret
2310	# check each var used in expression is defined
2311	defined = self.assgnVars.keys() + self.freeVars.keys()
2312	notfound = []
2313	for var in exprLblList:
2314	if var not in defined:
2315	notfound.append(var)
2316	if notfound:
2317	msg = 'Not all variables defined'
2318	msg1 = 'The following variables were not defined: '
2319	msg2 = ''
2320	for var in notfound:
2321	if msg: msg += ', '
2322	msg += var
2323	self.lastError = (msg1,' '+msg2)
2324	raise Exception(msg)
2325	return fxnpkgdict
2326
2327	def ParseExpression(self,expr):
2328	'''Parse an expression and return a dict of called functions and
2329	the variables used in the expression. Returns None in case an error
2330	is encountered. If packages are referenced in functions, they are loaded
2331	and the functions are looked up into the modules global
2332	workspace.
2333
2334	Note that no changes are made to the object other than
2335	saving an error message, so that this can be used for testing prior
2336	to the save.
2337
2338	:returns: a list of used variables
2339	'''
2340	self.lastError = ('','')
2341	import ast
2342	def FindFunction(f):
2343	'''Find the object corresponding to function f
2344	:param str f: a function name such as 'numpy.exp'
2345	:returns: (pkgdict,pkgobj) where pkgdict contains a dict
2346	that defines the package location(s) and where pkgobj
2347	defines the object associated with the function.
2348	If the function is not found, pkgobj is None.
2349	'''
2350	df = f.split('.')
2351	pkgdict = {}
2352	# no listed package, try in current namespace
2353	if len(df) == 1:
2354	try:
2355	fxnobj = eval(f)
2356	return pkgdict,fxnobj
2357	except (AttributeError, NameError):
2358	return None,None
2359	else:
2360	try:
2361	fxnobj = eval(f)
2362	pkgdict[df[0]] = eval(df[0])
2363	return pkgdict,fxnobj
2364	except (AttributeError, NameError):
2365	pass
2366	# includes a package, lets try to load the packages
2367	pkgname = ''
2368	path = sys.path
2369	for pkg in f.split('.')[:-1]: # if needed, descend down the tree
2370	if pkgname:
2371	pkgname += '.' + pkg
2372	else:
2373	pkgname = pkg
2374	fp = None
2375	try:
2376	fp, fppath,desc = imp.find_module(pkg,path)
2377	pkgobj = imp.load_module(pkg,fp,fppath,desc)
2378	pkgdict[pkgname] = pkgobj
2379	path = [fppath]
2380	except Exception as msg:
2381	print('load of '+pkgname+' failed with error='+str(msg))
2382	return {},None
2383	finally:
2384	if fp: fp.close()
2385	try:
2386	#print 'before',pkgdict.keys()
2387	fxnobj = eval(f,globals(),pkgdict)
2388	#print 'after 1',pkgdict.keys()
2389	#fxnobj = eval(f,pkgdict)
2390	#print 'after 2',pkgdict.keys()
2391	return pkgdict,fxnobj
2392	except:
2393	continue
2394	return None # not found
2395	def ASTtransverse(node,fxn=False):
2396	'''Transverse a AST-parsed expresson, compiling a list of variables
2397	referenced in the expression. This routine is used recursively.
2398
2399	:returns: varlist,fxnlist where
2400	varlist is a list of referenced variable names and
2401	fxnlist is a list of used functions
2402	'''
2403	varlist = []
2404	fxnlist = []
2405	if isinstance(node, list):
2406	for b in node:
2407	v,f = ASTtransverse(b,fxn)
2408	varlist += v
2409	fxnlist += f
2410	elif isinstance(node, ast.AST):
2411	for a, b in ast.iter_fields(node):
2412	if isinstance(b, ast.AST):
2413	if a == 'func':
2414	fxnlist += ['.'.join(ASTtransverse(b,True)[0])]
2415	continue
2416	v,f = ASTtransverse(b,fxn)
2417	varlist += v
2418	fxnlist += f
2419	elif isinstance(b, list):
2420	v,f = ASTtransverse(b,fxn)
2421	varlist += v
2422	fxnlist += f
2423	elif node.__class__.__name__ == "Name":
2424	varlist += [b]
2425	elif fxn and node.__class__.__name__ == "Attribute":
2426	varlist += [b]
2427	return varlist,fxnlist
2428	try:
2429	exprast = ast.parse(expr)
2430	except SyntaxError:
2431	s = ''
2432	import traceback
2433	for i in traceback.format_exc().splitlines()[-3:-1]:
2434	if s: s += "\n"
2435	s += str(i)
2436	self.lastError = ("Error parsing expression:",s)
2437	return
2438	# find the variables & functions
2439	v,f = ASTtransverse(exprast)
2440	varlist = sorted(list(set(v)))
2441	fxnlist = list(set(f))
2442	pkgdict = {}
2443	# check the functions are defined
2444	for fxn in fxnlist:
2445	fxndict,fxnobj = FindFunction(fxn)
2446	if not fxnobj:
2447	self.lastError = ("Error: Invalid function",fxn,
2448	"is not defined")
2449	return
2450	if not hasattr(fxnobj,'__call__'):
2451	self.lastError = ("Error: Not a function.",fxn,
2452	"cannot be called as a function")
2453	return
2454	pkgdict.update(fxndict)
2455	return varlist,pkgdict
2456
2457	def GetDepVar(self):
2458	'return the dependent variable, or None'
2459	return self.depVar
2460
2461	def SetDepVar(self,var):
2462	'Set the dependent variable, if used'
2463	self.depVar = var
2464	#==========================================================================
2465	class ExpressionCalcObj(object):
2466	'''An object used to evaluate an expression from a :class:`ExpressionObj`
2467	object.
2468
2469	:param ExpressionObj exprObj: a :class:`~ExpressionObj` expression object with
2470	an expression string and mappings for the parameter labels in that object.
2471	'''
2472	def __init__(self,exprObj):
2473	self.eObj = exprObj
2474	'The expression and mappings; a :class:`ExpressionObj` object'
2475	self.compiledExpr = None
2476	'The expression as compiled byte-code'
2477	self.exprDict = {}
2478	'''dict that defines values for labels used in expression and packages
2479	referenced by functions
2480	'''
2481	self.lblLookup = {}
2482	'''Lookup table that specifies the expression label name that is
2483	tied to a particular GSAS-II parameters in the parmDict.
2484	'''
2485	self.fxnpkgdict = {}
2486	'''a dict with references to packages needed to
2487	find functions referenced in the expression.
2488	'''
2489	self.varLookup = {}
2490	'''Lookup table that specifies the GSAS-II variable(s)
2491	indexed by the expression label name. (Used for only for diagnostics
2492	not evaluation of expression.)
2493	'''
2494	self.su = None
2495	'''Standard error evaluation where supplied by the evaluator
2496	'''
2497	# Patch: for old-style expressions with a (now removed step size)
2498	for v in self.eObj.assgnVars:
2499	if not isinstance(self.eObj.assgnVars[v], basestring):
2500	self.eObj.assgnVars[v] = self.eObj.assgnVars[v][0]
2501	self.parmDict = {}
2502	'''A copy of the parameter dictionary, for distance and angle computation
2503	'''
2504
2505	def SetupCalc(self,parmDict):
2506	'''Do all preparations to use the expression for computation.
2507	Adds the free parameter values to the parameter dict (parmDict).
2508	'''
2509	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
2510	return
2511	self.fxnpkgdict = self.eObj.CheckVars()
2512	# all is OK, compile the expression
2513	self.compiledExpr = compile(self.eObj.expression,'','eval')
2514
2515	# look at first value in parmDict to determine its type
2516	parmsInList = True
2517	for key in parmDict:
2518	val = parmDict[key]
2519	if isinstance(val, basestring):
2520	parmsInList = False
2521	break
2522	try: # check if values are in lists
2523	val = parmDict[key][0]
2524	except (TypeError,IndexError):
2525	parmsInList = False
2526	break
2527
2528	# set up the dicts needed to speed computations
2529	self.exprDict = {}
2530	self.lblLookup = {}
2531	self.varLookup = {}
2532	for v in self.eObj.freeVars:
2533	varname = self.eObj.freeVars[v][0]
2534	varname = "::" + varname.lstrip(':').replace(' ','_').replace(':',';')
2535	self.lblLookup[varname] = v
2536	self.varLookup[v] = varname
2537	if parmsInList:
2538	parmDict[varname] = [self.eObj.freeVars[v][1],self.eObj.freeVars[v][2]]
2539	else:
2540	parmDict[varname] = self.eObj.freeVars[v][1]
2541	self.exprDict[v] = self.eObj.freeVars[v][1]
2542	for v in self.eObj.assgnVars:
2543	varname = self.eObj.assgnVars[v]
2544	if '*' in varname:
2545	varlist = LookupWildCard(varname,parmDict.keys())
2546	if len(varlist) == 0:
2547	raise Exception,"No variables match "+str(v)
2548	for var in varlist:
2549	self.lblLookup[var] = v
2550	if parmsInList:
2551	self.exprDict[v] = np.array([parmDict[var][0] for var in varlist])
2552	else:
2553	self.exprDict[v] = np.array([parmDict[var] for var in varlist])
2554	self.varLookup[v] = [var for var in varlist]
2555	elif varname in parmDict:
2556	self.lblLookup[varname] = v
2557	self.varLookup[v] = varname
2558	if parmsInList:
2559	self.exprDict[v] = parmDict[varname][0]
2560	else:
2561	self.exprDict[v] = parmDict[varname]
2562	else:
2563	self.exprDict[v] = None
2564	# raise Exception,"No value for variable "+str(v)
2565	self.exprDict.update(self.fxnpkgdict)
2566
2567	def UpdateVars(self,varList,valList):
2568	'''Update the dict for the expression with a set of values
2569	:param list varList: a list of variable names
2570	:param list valList: a list of corresponding values
2571	'''
2572	for var,val in zip(varList,valList):
2573	self.exprDict[self.lblLookup.get(var,'undefined: '+var)] = val
2574
2575	def UpdateDict(self,parmDict):
2576	'''Update the dict for the expression with values in a dict
2577	:param list parmDict: a dict of values some of which may be in use here
2578	'''
2579	if self.eObj.expression.startswith('Dist') or self.eObj.expression.startswith('Angle'):
2580	self.parmDict = parmDict
2581	return
2582	for var in parmDict:
2583	if var in self.lblLookup:
2584	self.exprDict[self.lblLookup[var]] = parmDict[var]
2585
2586	def EvalExpression(self):
2587	'''Evaluate an expression. Note that the expression
2588	and mapping are taken from the :class:`ExpressionObj` expression object
2589	and the parameter values were specified in :meth:`SetupCalc`.
2590	:returns: a single value for the expression. If parameter
2591	values are arrays (for example, from wild-carded variable names),
2592	the sum of the resulting expression is returned.
2593
2594	For example, if the expression is ``'A*B'``,
2595	where A is 2.0 and B maps to ``'1::Afrac:*'``, which evaluates to::
2596
2597	[0.5, 1, 0.5]
2598
2599	then the result will be ``4.0``.
2600	'''
2601	self.su = None
2602	if self.eObj.expression.startswith('Dist'):
2603	# GSASIIpath.IPyBreak()
2604	dist = G2mth.CalcDist(self.eObj.distance_dict, self.eObj.distance_atoms, self.parmDict)
2605	return dist
2606	elif self.eObj.expression.startswith('Angle'):
2607	angle = G2mth.CalcAngle(self.eObj.angle_dict, self.eObj.angle_atoms, self.parmDict)
2608	return angle
2609	if self.compiledExpr is None:
2610	raise Exception,"EvalExpression called before SetupCalc"
2611	try:
2612	val = eval(self.compiledExpr,globals(),self.exprDict)
2613	except TypeError:
2614	val = None
2615	if not np.isscalar(val):
2616	val = np.sum(val)
2617	return val
2618
2619	class G2Exception(Exception):
2620	def __init__(self,msg):
2621	self.msg = msg
2622	def __str__(self):
2623	return repr(self.msg)
2624
2625	def CreatePDFitems(G2frame,PWDRtree,ElList,Qlimits,numAtm=1,FltBkg=0,PDFnames=[]):
2626	'''Create and initialize a new set of PDF tree entries
2627
2628	:param Frame G2frame: main GSAS-II tree frame object
2629	:param str PWDRtree: name of PWDR to be used to create PDF item
2630	:param dict ElList: data structure with composition
2631	:param list Qlimits: Q limits to be used for computing the PDF
2632	:param float numAtm: no. atom in chemical formula
2633	:param float FltBkg: flat background value
2634	:param list PDFnames: previously used PDF names
2635
2636	:returns: the Id of the newly created PDF entry
2637	'''
2638	PDFname = 'PDF '+PWDRtree[4:] # this places two spaces after PDF, which is needed is some places
2639	if PDFname in PDFnames:
2640	print('Skipping, entry already exists: '+PDFname)
2641	return None
2642	#PDFname = MakeUniqueLabel(PDFname,PDFnames)
2643	Id = G2frame.PatternTree.AppendItem(parent=G2frame.root,text=PDFname)
2644	Data = {
2645	'Sample':{'Name':PWDRtree,'Mult':1.0},
2646	'Sample Bkg.':{'Name':'','Mult':-1.0,'Refine':False},
2647	'Container':{'Name':'','Mult':-1.0,'Refine':False},
2648	'Container Bkg.':{'Name':'','Mult':-1.0},'ElList':ElList,
2649	'Geometry':'Cylinder','Diam':1.0,'Pack':0.50,'Form Vol':10.0*numAtm,'Flat Bkg':FltBkg,
2650	'DetType':'Area detector','ObliqCoeff':0.2,'Ruland':0.025,'QScaleLim':Qlimits,
2651	'Lorch':False,'BackRatio':0.0,'Rmax':100.,'noRing':False,'IofQmin':1.0,
2652	'I(Q)':[],'S(Q)':[],'F(Q)':[],'G(R)':[]}
2653	G2frame.PatternTree.SetItemPyData(G2frame.PatternTree.AppendItem(Id,text='PDF Controls'),Data)
2654	G2frame.PatternTree.SetItemPyData(G2frame.PatternTree.AppendItem(Id,text='PDF Peaks'),
2655	{'Limits':[1.,5.],'Background':[2,[0.,-0.2*np.pi],False],'Peaks':[]})
2656	return Id
2657
2658
2659	if __name__ == "__main__":
2660	# test equation evaluation
2661	def showEQ(calcobj):
2662	print 50*'='
2663	print calcobj.eObj.expression,'=',calcobj.EvalExpression()
2664	for v in sorted(calcobj.varLookup):
2665	print " ",v,'=',calcobj.exprDict[v],'=',calcobj.varLookup[v]
2666	# print ' Derivatives'
2667	# for v in calcobj.derivStep.keys():
2668	# print ' d(Expr)/d('+v+') =',calcobj.EvalDeriv(v)
2669
2670	obj = ExpressionObj()
2671
2672	obj.expression = "A*np.exp(B)"
2673	obj.assgnVars = {'B': '0::Afrac:1'}
2674	obj.freeVars = {'A': [u'A', 0.5, True]}
2675	#obj.CheckVars()
2676	calcobj = ExpressionCalcObj(obj)
2677
2678	obj1 = ExpressionObj()
2679	obj1.expression = "A*np.exp(B)"
2680	obj1.assgnVars = {'B': '0::Afrac:*'}
2681	obj1.freeVars = {'A': [u'Free Prm A', 0.5, True]}
2682	#obj.CheckVars()
2683	calcobj1 = ExpressionCalcObj(obj1)
2684
2685	obj2 = ExpressionObj()
2686	obj2.distance_stuff = np.array([[0,1],[1,-1]])
2687	obj2.expression = "Dist(1,2)"
2688	GSASIIpath.InvokeDebugOpts()
2689	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
2690	calcobj2 = ExpressionCalcObj(obj2)
2691	calcobj2.SetupCalc(parmDict2)
2692	showEQ(calcobj2)
2693
2694	parmDict1 = {'0::Afrac:0':1.0, '0::Afrac:1': 1.0}
2695	print '\nDict = ',parmDict1
2696	calcobj.SetupCalc(parmDict1)
2697	showEQ(calcobj)
2698	calcobj1.SetupCalc(parmDict1)
2699	showEQ(calcobj1)
2700
2701	parmDict2 = {'0::Afrac:0':[0.0,True], '0::Afrac:1': [1.0,False]}
2702	print 'Dict = ',parmDict2
2703	calcobj.SetupCalc(parmDict2)
2704	showEQ(calcobj)
2705	calcobj1.SetupCalc(parmDict2)
2706	showEQ(calcobj1)
2707	calcobj2.SetupCalc(parmDict2)
2708	showEQ(calcobj2)

Note: See TracBrowser for help on using the repository browser.

Download in other formats: