source: trunk/GSASIImapvars.py @ 1243

Last change on this file since 1243 was 1243, checked in by toby, 8 years ago

fix constraints use in sequential refinements

  • Property svn:eol-style set to native
  • Property svn:keywords set to Date Author Revision URL Id
File size: 49.2 KB
Line 
1# -*- coding: utf-8 -*-
2########### SVN repository information ###################
3# $Date: 2014-03-10 19:06:56 +0000 (Mon, 10 Mar 2014) $
4# $Author: toby $
5# $Revision: 1243 $
6# $URL: trunk/GSASIImapvars.py $
7# $Id: GSASIImapvars.py 1243 2014-03-10 19:06:56Z toby $
8########### SVN repository information ###################
9"""
10*GSASIImapvars: Parameter constraints*
11======================================
12
13Module to implements algebraic contraints, parameter redefinition
14and parameter simplification contraints.
15
16Parameter redefinition (new vars) is done by creating one or more relationships
17between a set of parameters
18
19::
20
21   Mx1 * Px + My1 * Py +...
22   Mx2 * Px + Mz2 * Pz + ...
23
24where Pj is a parameter name and Mjk is a constant.
25
26Constant constraint Relations can also be supplied in the form of an equation:
27
28::
29
30  nx1 * Px + ny1 * Py +... = C1
31
32where Cn is a constant. These equations define an algebraic
33constrant.
34
35Parameters can also be "fixed" (held), which prevents them from being refined.
36
37All of the above three cases are input using routines
38GroupConstraints and GenerateConstraints. The input consists of a list of
39relationship dictionaries:
40
41.. code-block:: python
42
43    constrDict = [
44         {'0:12:Scale': 2.0, '0:14:Scale': 4.0, '0:13:Scale': 3.0, '0:0:Scale': 0.5},
45         {'2::C(10,6,1)': 1.0, '1::C(10,6,1)': 1.0},
46         {'0::A0': 0.0}]
47    fixedList = ['5.0', None, '0']
48
49Where the dictionary defines the first part of an expression and the corresponding fixedList
50item is either None (for parameter redefinition) or the constant value, for a constant
51constraint equation. A dictionary that contains a single term defines a variable that
52will be fixed (held). The multiplier and the fixedList value in this case are ignored.
53
54Parameters can also be equivalenced or "slaved" to another parameter, such that one
55(independent) parameter is equated to several (now dependent) parameters. In
56algebraic form this is:
57
58::
59
60   P0 = M1 * P1 = M2 * P2 = ...
61
62Thus parameters P0, P1 and P2,... are linearly equivalent. Routine StoreEquivalence is
63used to specify these equivalences.
64
65Parameter redefinition (new vars) describes a new, independent, parameter, which
66is defined in terms of dependent parameters that are defined in the
67Model, while fixed constrained relations effectively reduce the complexity
68of the Model by removing a degree of freedom. It is possible for a parameter to appear
69in both a parameter redefinition expression and a fixed constraint equation, but a
70parameter cannot be used a parameter equivalance cannot be used elsewhere (not fixed,
71constrained or redefined). Likewise a fixed parameter cannot be used elsewhere (not
72equivalanced, constrained or redefined).
73
74Relationships are grouped so that a set of dependent parameters appear
75in only one group (done in routine GroupConstraints.) Note that if a
76group contains relationships/equations that involve N dependent
77parameters, there must exist N-C new parameters, where C is the number
78of contraint equations in the group. Routine GenerateConstraints takes
79the output from GroupConstraints and generates the
80"missing" relationships and saves that information in the module's
81global variables. Each generated parameter is named sequentially using paramPrefix.
82
83A list of parameters that will be varied is specified as input to GenerateConstraints
84(varyList). A fixed parameter will simply be removed from this list preventing that
85parameter from being varied. Note that all parameters in a constraint relationship
86must specified as varied (appear in varyList) or none can be varied. This is
87checked in GenerateConstraints. Likewise, if all parameters in a constraint are
88not referenced in a refinement, the constraint is ignored, but if some parameters
89in a constraint group are not referenced in a refinement, but others are this
90constitutes and error.
91
92* When a new variable is created, the variable is assigned the name associated
93  in the constraint definition or it is assigned a default name of form
94  ``::constr<n>`` (see paramPrefix). The vary setting for variables used in the
95  constraint are ignored.
96  Note that any generated "missing" relations are not varied. Only
97  the input relations can be are varied.
98 
99* If all parameters in a fixed constraint equation are varied, the generated "missing"
100  relations in the group are all varied. This provides the N-C degrees of freedom.
101
102*External Routines*
103-------------------
104
105To define a set of constrained and unconstrained relations, one
106defines a list of dictionary defining constraint parameters and their
107values, a list of fixed values for each constraint and a list of
108parameters to be varied. In addition, one uses
109:func:`StoreEquivalence` to define parameters that are equivalent. One
110can then use :func:`CheckConstraints` to check that the input is
111internally consistent and finally :func:`GroupConstraints` and
112:func:`GenerateConstraints` to generate the internally used
113tables. Routines :func:`Map2Dict` is used to initialize the parameter
114dictionary and :func:`Dict2Map`, :func:`Dict2Deriv`, and
115:func:`ComputeDepESD` are used to apply constraints. Routine
116:func:`VarRemapShow` is used to print out the constraint information,
117as stored by :func:`GenerateConstraints`.
118
119:func:`InitVars`
120  This is optionally used to clear out all defined previously defined constraint information
121 
122:func:`StoreEquivalence`
123  To implement parameter redefinition, one calls StoreEquivalence. This should be called for every set of
124  equivalence relationships. There is no harm in using StoreEquivalence with the same independent variable:
125
126  .. code-block:: python
127
128       StoreEquivalence('x',('y',))
129       StoreEquivalence('x',('z',))
130
131  or equivalently
132
133  .. code-block:: python
134
135       StoreEquivalence('x',('y','z'))
136
137  The latter will run more efficiently. Note that mixing independent and dependent variables is
138  problematic. This is not allowed:
139
140  .. code-block:: python
141
142        StoreEquivalence('x',('y',))
143        StoreEquivalence('y',('z',))
144       
145  Use StoreEquivalence before calling GenerateConstraints or CheckConstraints
146
147:func:`CheckConstraints`
148   To check that input in internally consistent, use CheckConstraints
149
150:func:`Map2Dict`
151   To determine values for the parameters created in this module, one
152   calls Map2Dict. This will not apply contraints.
153
154:func:`Dict2Map`
155   To take values from the new independent parameters and constraints,
156   one calls Dict2Map. This will apply contraints.
157
158:func:`Dict2Deriv`
159   Use Dict2Deriv to determine derivatives on independent parameters
160   from those on dependent ones
161
162:func:`ComputeDepESD`     
163   Use ComputeDepESD to compute uncertainties on dependent variables
164
165:func:`VarRemapShow`
166   To show a summary of the parameter remapping, one calls VarRemapShow
167
168*Global Variables*
169------------------
170
171dependentParmList:
172   contains a list by group of lists of
173   parameters used in the group. Note that parameters listed in
174   dependentParmList should not be refined as they will not affect
175   the model
176
177indParmList:
178     a list of groups of Independent parameters defined in
179     each group. This contains both parameters used in parameter
180     redefinitions as well as names of generated new parameters.
181
182fixedVarList:
183     a list of variables that have been 'fixed'
184     by defining them as equal to a constant (::var: = 0). Note that
185     the constant value is ignored at present. These variables are
186     later removed from varyList which prevents them from being refined.
187     Unlikely to be used externally.
188
189arrayList:
190     a list by group of relationship matrices to relate
191     parameters in dependentParmList to those in indParmList. Unlikely
192     to be used externally.
193
194invarrayList:
195     a list by group of relationship matrices to relate
196     parameters in indParmList to those in dependentParmList. Unlikely
197     to be used externally.
198
199fixedDict:
200     a dictionary containing the fixed values corresponding
201     to parameter equations.  The dict key is an ascii string, but the
202     dict value is a float.  Unlikely to be used externally.
203
204*Routines*
205----------
206
207Note that parameter names in GSAS-II are strings of form ``<ph>:<hst>:<nam>``
208
209"""
210
211import numpy as np
212import GSASIIpath
213GSASIIpath.SetVersionNumber("$Revision: 1243 $")
214# data used for constraints;
215debug = False # turns on printing as constraint input is processed
216# note that constraints are stored listed by contraint groups, where each constraint
217# group contains those parameters that must be handled together
218dependentParmList = [] # contains a list of parameters in each group
219# note that parameters listed in dependentParmList should not be refined
220arrayList = [] # a list of of relationship matrices
221invarrayList = [] # a list of inverse relationship matrices
222indParmList = [] # a list of names for the new parameters
223fixedDict = {} # a dictionary containing the fixed values corresponding to defined parameter equations
224               # key is original ascii string, value is float
225fixedVarList = [] # List of variables that should not be refined
226
227# prefix for parameter names
228paramPrefix = "::constr"
229consNum = 0 # number of the next constraint to be created
230
231def InitVars():
232    '''Initializes all constraint information'''
233    global dependentParmList,arrayList,invarrayList,indParmList,fixedDict,consNum
234    dependentParmList = [] # contains a list of parameters in each group
235    arrayList = [] # a list of of relationship matrices
236    invarrayList = [] # a list of inverse relationship matrices
237    indParmList = [] # a list of names for the new parameters
238    fixedDict = {} # a dictionary containing the fixed values corresponding to defined parameter equations
239    consNum = 0 # number of the next constraint to be created
240    fixedVarList = []
241
242def VarKeys(constr):
243    """Finds the keys in a constraint that represent variables
244    e.g. eliminates any that start with '_'
245
246    :param dict constr: a single constraint entry of form::
247
248        {'var1': mult1, 'var2': mult2,... '_notVar': val,...}
249
250        (see :func:`GroupConstraints`)
251    :returns: a list of keys where any keys beginning with '_' are
252      removed.
253    """
254    return [i for i in constr.keys() if not i.startswith('_')]
255
256
257def GroupConstraints(constrDict):
258    """divide the constraints into groups that share no parameters.
259
260    :param dict constrDict: a list of dicts defining relationships/constraints
261
262    ::
263   
264       constrDict = [{<constr1>}, {<constr2>}, ...]
265
266    where {<constr1>} is {'var1': mult1, 'var2': mult2,... }
267
268    :returns: two lists of lists:
269   
270      * a list of grouped contraints where each constraint grouped containts a list
271        of indices for constraint constrDict entries
272      * a list containing lists of parameter names contained in each group
273     
274      """
275    assignedlist = [] # relationships that have been used
276    groups = [] # contains a list of grouplists
277    ParmList = []
278    for i,consi in enumerate(constrDict):
279        if i in assignedlist: continue # already in a group, skip
280        # starting a new group
281        grouplist = [i,]
282        assignedlist.append(i)
283        groupset = set(VarKeys(consi))
284        changes = True # always loop at least once
285        while(changes): # loop until we can't find anything to add to the current group
286            changes = False # but don't loop again unless we find something
287            for j,consj in enumerate(constrDict):
288                if j in assignedlist: continue # already in a group, skip
289                if len(set(VarKeys(consj)) & groupset) > 0: # true if this needs to be added
290                    changes = True
291                    grouplist.append(j)
292                    assignedlist.append(j)
293                    groupset = groupset | set(VarKeys(consj))
294        group = sorted(grouplist)
295        varlist = sorted(list(groupset))
296        groups.append(group)
297        ParmList.append(varlist)
298    return groups,ParmList
299
300def CheckConstraints(varyList,constrDict,fixedList):
301    '''Takes a list of relationship entries comprising a group of
302    constraints and checks for inconsistencies such as conflicts in
303    parameter/variable definitions and or inconsistently varied parameters.
304
305    :param list varyList: a list of parameters names that will be varied
306
307    :param dict constrDict: a list of dicts defining relationships/constraints
308      (as created in :func:`GSASIIstrIO.ProcessConstraints` and
309      documented in :func:`GroupConstraints`)
310
311    :param list fixedList: a list of values specifying a fixed value for each
312      dict in constrDict. Values are either strings that can be converted to
313      floats or ``None`` if the constraint defines a new parameter rather
314      than a constant.
315
316    :returns: two strings:
317
318      * the first lists conflicts internal to the specified constraints
319      * the second lists conflicts where the varyList specifies some
320        parameters in a constraint, but not all
321       
322      If there are no errors, both strings will be empty
323    '''
324    import re
325    global dependentParmList,arrayList,invarrayList,indParmList,consNum
326    errmsg = ''
327    warnmsg = ''
328    fixVlist = []
329    # process fixed variables (holds)
330    for cdict in constrDict:
331        # N.B. No "_" names in holds
332        if len(cdict) == 1:
333            fixVlist.append(cdict.keys()[0])
334   
335    # process equivalences: make a list of dependent and independent vars
336    #    and check for repeated uses (repetition of a parameter as an
337    #    independent var is OK)
338    indepVarList = []
339    depVarList = []
340    multdepVarList = []
341    for varlist,mapvars,multarr,invmultarr in zip(
342        dependentParmList,indParmList,arrayList,invarrayList):
343        if multarr is None: # an equivalence
344            zeromult = False
345            for mv in mapvars:
346                varied = 0
347                notvaried = ''
348                if mv in varyList:
349                    varied += 1
350                else:
351                    if notvaried: notvaried += ', '
352                    notvaried += mv
353                if mv not in indepVarList: indepVarList.append(mv)
354                for v,m in zip(varlist,invmultarr):
355                    if m == 0: zeromult = True
356                    if v in varyList:
357                        varied += 1
358                    else:
359                        if notvaried: notvaried += ', '
360                        notvaried += v
361                    if v in depVarList:
362                        multdepVarList.append(v)
363                    else:
364                        depVarList.append(v)
365            if varied > 0 and varied != len(varlist)+1:
366                warnmsg += "\nNot all variables refined in equivalence:\n\t"
367                s = ""
368                for v in varlist:
369                    if s != "": s+= " & "
370                    s += str(v)           
371                warnmsg += str(mv) + " => " + s
372                warnmsg += '\nNot refined: ' + notvaried + '\n'
373            if zeromult:
374                errmsg += "\nZero multiplier is invalid in equivalence:\n\t"
375                s = ""
376                for v in varlist:
377                    if s != "": s+= " & "
378                    s += str(v)           
379                errmsg += str(mv) + " => " + s + '\n'
380
381    # check for errors:
382    inboth = set(indepVarList).intersection(set(depVarList))
383    if len(inboth) > 0:
384        errmsg += "\nThe following parameters(s) are used as both dependent and independent variables in Equivalence relations:\n"
385        s = ''
386        for var in sorted(inboth):
387            if s != "": s+= ", "
388            s += str(var)
389        errmsg += '\t'+ s + '\n'
390    if len(multdepVarList) > 0:
391        errmsg += "\nThe following parameters(s) are used in multiple Equivalence relations as dependent variables:\n"
392        s = ''
393        for var in sorted(set(multdepVarList)):
394            if s != "": s+= ", "
395            s += str(var)           
396        errmsg += '\t'+ s + '\n'
397    equivVarList = list(set(indepVarList).union(set(depVarList)))
398    #print 'equivVarList',equivVarList
399    inboth = set(fixVlist).intersection(set(equivVarList))
400    if len(inboth) > 0:
401        errmsg += "\nThe following parameter(s) are used in both Equivalence and Fixed constraints:\n"
402        s = ''
403        for var in sorted(inboth):
404            if s != "": s+= ", "
405            s += str(var)
406        errmsg += '\t'+ s + '\n'
407
408    groups,parmlist = GroupConstraints(constrDict)
409    # scan through parameters in each relationship. Are all varied? If only some are
410    # varied, create a warning message.
411    for group,varlist in zip(groups,parmlist):
412        if len(varlist) == 1: continue
413        for rel in group:
414            varied = 0
415            notvaried = ''
416            for var in constrDict[rel]:
417                if var.startswith('_'): continue
418                if not re.match('[0-9]*:[0-9]*:',var):
419                    warnmsg += "\nVariable "+str(var)+" does not begin with a ':'"
420                if var in varyList:
421                    varied += 1
422                else:
423                    if notvaried: notvaried += ', '
424                    notvaried += var
425                if var in fixVlist:
426                    errmsg += '\nParameter '+var+" is Fixed and used in a constraint:\n\t"
427                    errmsg += _FormatConstraint(constrDict[rel],fixedList[rel])+"\n"
428            if varied > 0 and varied != len(VarKeys(constrDict[rel])):
429                warnmsg += "\nNot all variables refined in constraint:\n\t"
430                warnmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
431                warnmsg += '\nNot refined: ' + notvaried + '\n'
432    if errmsg or warnmsg:
433        return errmsg,warnmsg
434
435    # now look for process each group and create the relations that are needed to form
436    # non-singular square matrix
437    for group,varlist in zip(groups,parmlist):
438        if len(varlist) == 1: continue # a constraint group with a single variable can be ignored
439        if len(varlist) < len(group): # too many relationships -- no can do
440            errmsg += "\nOver-constrained input. "
441            errmsg += "There are more constraints " + str(len(group))
442            errmsg += "\n\tthan variables " + str(len(varlist)) + "\n"
443            for rel in group:
444                errmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
445                errmsg += "\n"
446                continue
447        try:
448            multarr = _FillArray(group,constrDict,varlist)
449            _RowEchelon(len(group),multarr,varlist)
450        except:
451            errmsg += "\nSingular input. "
452            errmsg += "There are internal inconsistencies in these constraints\n"
453            for rel in group:
454                errmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
455                errmsg += "\n"
456            continue
457        try:
458            multarr = _FillArray(group,constrDict,varlist,FillDiagonals=True)
459            GramSchmidtOrtho(multarr,len(group))
460        except:
461            errmsg += "\nUnexpected singularity with constraints (in Gram-Schmidt)\n"
462            for rel in group:
463                errmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
464                errmsg += "\n"
465            continue
466        mapvar = []
467        group = group[:]
468        # scan through all generated and input variables
469        # Check again for inconsistent variable use
470        # for new variables -- where varied and unvaried parameters get grouped
471        # together. I don't think this can happen when not flagged before, but
472        # it does not hurt to check again.
473        for i in range(len(varlist)):
474            varied = 0
475            notvaried = ''
476            if len(group) > 0:
477                rel = group.pop(0)
478                fixedval = fixedList[rel]
479                for var in VarKeys(constrDict[rel]):
480                    if var in varyList:
481                        varied += 1
482                    else:
483                        if notvaried: notvaried += ', '
484                        notvaried += var
485            else:
486                fixedval = None
487            if fixedval is None:
488                varname = paramPrefix + str(consNum) # assign a name to a variable
489                mapvar.append(varname)
490                consNum += 1
491            else:
492                mapvar.append(fixedval)
493            if varied > 0 and notvaried != '':
494                warnmsg += "\nNot all variables refined in generated constraint"
495                warnmsg += '\nPlease report this unexpected error\n'
496                for rel in group:
497                    warnmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
498                    warnmsg += "\n"
499                warnmsg += '\n\tNot refined: ' + notvaried + '\n'
500        try:
501            np.linalg.inv(multarr)           
502        except:
503            errmsg += "\nSingular input. "
504            errmsg += "The following constraints are not "
505            errmsg += "linearly independent\n\tor do not "
506            errmsg += "allow for generation of a non-singular set\n"
507            errmsg += 'Please report this unexpected error\n'
508            for rel in group:
509                errmsg += _FormatConstraint(constrDict[rel],fixedList[rel])
510                errmsg += "\n"
511    return errmsg,warnmsg
512
513def GenerateConstraints(groups,parmlist,varyList,constrDict,fixedList,parmDict=None):
514    '''Takes a list of relationship entries comprising a group of
515    constraints and builds the relationship lists and their inverse
516    and stores them in global variables Also checks for internal
517    conflicts or inconsistencies in parameter/variable definitions.
518
519    :param list groups: a list of grouped contraints where each constraint
520      grouped containts a list of indices for constraint constrDict entries,
521      created in :func:`GroupConstraints` (returned as 1st value)
522
523    :param list parmlist: a list containing lists of parameter names
524      contained in each group, created in :func:`GroupConstraints`
525      (returned as 2nd value)
526
527    :param list varyList: a list of parameters names (strings of form
528      ``<ph>:<hst>:<nam>``) that will be varied
529   
530    :param dict constrDict: a list of dicts defining relationships/constraints
531      (as defined in :func:`GroupConstraints`)
532
533    :param list fixedList: a list of values specifying a fixed value for each
534      dict in constrDict. Values are either strings that can be converted to
535      floats, float values or None if the constraint defines a new parameter.
536     
537    :param dict constrDict: a list of dicts defining relationships/constraints.
538
539    '''
540    global dependentParmList,arrayList,invarrayList,indParmList,consNum
541    msg = ''
542
543    # process fixed (held) variables
544    for cdict in constrDict:
545        if len(cdict) == 1:
546            fixedVarList.append(cdict.keys()[0])
547   
548    # process equivalences: make a list of dependent and independent vars
549    #    and check for repeated uses (repetition of a parameter as an
550    #    independent var is OK [A=B; A=C], but chaining: [A=B; B=C] is not good)
551    indepVarList = []
552    depVarList = []
553    multdepVarList = []
554    for varlist,mapvars,multarr,invmultarr in zip(       # process equivalences
555        dependentParmList,indParmList,arrayList,invarrayList):
556        if multarr is None: # true only if an equivalence
557            zeromult = False
558            for mv in mapvars:
559                #s = ''
560                varied = 0
561                notvaried = ''
562                if mv in varyList:
563                    varied += 1
564                else:
565                    if notvaried: notvaried += ', '
566                    notvaried += mv
567                if mv not in indepVarList: indepVarList.append(mv)
568                if parmDict is not None and mv not in parmDict:
569                    msg += "\nCannot equivalence to variable "+str(mv)+". Not defined in this refinement"
570                    continue
571                for v,m in zip(varlist,invmultarr):
572                    if parmDict is not None and v not in parmDict:
573                        print "Dropping equivalence for variable "+str(v)+". Not defined in this refinement"
574                        continue
575                    if m == 0: zeromult = True
576                    if v in varyList:
577                        varied += 1
578                    else:
579                        if notvaried: notvaried += ', '
580                        notvaried += v
581                    if v in depVarList:
582                        multdepVarList.append(v)
583                    else:
584                        depVarList.append(v)
585            if varied > 0 and varied != len(varlist)+1:
586                msg += "\nNot all variables refined in equivalence:\n\t"
587                s = ""
588                for v in varlist:
589                    if s != "": s+= " & "
590                    s += str(v)           
591                msg += str(mv) + " => " + s
592                msg += '\nNot refined: ' + notvaried + '\n'
593            if zeromult:
594                msg += "\nZero multiplier is invalid in equivalence:\n\t"
595                s = ""
596                for v in varlist:
597                    if s != "": s+= " & "
598                    s += str(v)           
599                msg += str(mv) + " => " + s + '\n'
600
601    #print 'indepVarList',indepVarList
602    #print 'depVarList',depVarList
603    # check for errors:
604    inboth = set(indepVarList).intersection(set(depVarList))
605    if len(inboth) > 0:
606        msg += "\nThe following parameters(s) are used as both dependent and independent variables in Equivalence relations:\n"
607        s = ''
608        for var in sorted(inboth):
609            if s != "": s+= ", "
610            s += str(var)
611        msg += '\t'+ s + '\n'
612    if len(multdepVarList) > 0:
613        msg += "\nThe following parameters(s) are used in multiple Equivalence relations as dependent variables:\n"
614        s = ''
615        for var in sorted(set(multdepVarList)):
616            if s != "": s+= ", "
617            s += str(var)           
618        msg += '\t'+ s + '\n'
619    equivVarList = list(set(indepVarList).union(set(depVarList)))
620
621    # scan through parameters in each relationship. Are all varied? If only some are
622    # varied, create an error message.
623    for group,varlist in zip(groups,parmlist):
624        if len(varlist) == 1: continue
625        for rel in group:
626            varied = 0
627            notvaried = ''
628            unused = 0
629            notused = ''
630            for var in constrDict[rel]:
631                if var.startswith('_'): continue
632                if parmDict is not None and var not in parmDict:
633                    unused += 1
634                    if notvaried: notused += ', '
635                    notused += var
636                if var in varyList:
637                    varied += 1
638                else:
639                    if notvaried: notvaried += ', '
640                    notvaried += var
641                if var in fixedVarList:
642                    msg += '\nError: parameter '+var+" is Fixed and used in a constraint:\n\t"
643                    msg += _FormatConstraint(constrDict[rel],fixedList[rel])+"\n"
644            #if unused > 0:# and unused != len(VarKeys(constrDict[rel])):
645            if unused > 0 and unused != len(VarKeys(constrDict[rel])):
646                msg += "\nSome (but not all) variables in constraint are not defined:\n\t"
647                msg += _FormatConstraint(constrDict[rel],fixedList[rel])
648                msg += '\nNot used: ' + notused + '\n'
649            if varied > 0 and varied != len(VarKeys(constrDict[rel])):
650                msg += "\nNot all variables refined in constraint:\n\t"
651                msg += _FormatConstraint(constrDict[rel],fixedList[rel])
652                msg += '\nNot refined: ' + notvaried + '\n'
653    # if there were errors found, go no farther
654    if msg:
655        print ' *** ERROR in constraint definitions! ***'
656        print msg
657        raise Exception
658               
659    # now process each group and create the relations that are needed to form
660    # a non-singular square matrix
661    # If all are varied and this is a constraint equation, then set VaryFree flag
662    # so that the newly created relationships will be varied
663    for group,varlist in zip(groups,parmlist):
664        if len(varlist) == 1: continue
665        # for constraints, if all included variables are refined,
666        # set the VaryFree flag, and remaining degrees of freedom will be
667        # varied (since consistency was checked, if any one variable is
668        # refined, then assume that all are)
669        varsList = [] # make a list of all the referenced variables as well
670        VaryFree = False
671        for rel in group:
672            varied = 0
673            unused = 0
674            for var in VarKeys(constrDict[rel]):
675                if parmDict is not None and var not in parmDict:
676                    unused += 1                   
677                if var not in varsList: varsList.append(var)
678                if var in varyList: varied += 1
679            if fixedList[rel] is not None and varied > 0:
680                VaryFree = True
681        if len(varlist) < len(group): # too many relationships -- no can do
682            msg = 'too many relationships'
683            break
684        # Since we checked before, if any variables are unused, then all must be.
685        # If so, this set of relationships can be ignored
686        if unused:
687            if debug: print('Constraint ignored (all variables undefined)')
688            if debug: print ('    '+_FormatConstraint(constrDict[rel],fixedList[rel]))
689            continue
690        # fill in additional degrees of freedom
691        try:
692            arr = _FillArray(group,constrDict,varlist)
693            _RowEchelon(len(group),arr,varlist)
694            constrArr = _FillArray(group,constrDict,varlist,FillDiagonals=True)
695            GramSchmidtOrtho(constrArr,len(group))
696        except:
697            msg = 'Singular relationships'
698            break
699        mapvar = []
700        group = group[:]
701        # scan through all generated and input relationships, we need to add to the varied list
702        # all the new parameters where VaryFree has been set or where a New Var is varied.
703        #
704        # If a group does not contain any fixed values (constraint equations)
705        # and nothing in the group is varied, drop this group, so that the
706        # dependent parameters can be refined individually.
707        unused = True
708        for i in range(len(varlist)):
709            if len(group) > 0: # get the original equation reference
710                rel = group.pop(0)
711                fixedval = fixedList[rel]
712                varyflag = constrDict[rel].get('_vary',False)
713                varname = constrDict[rel].get('_name','')
714            else: # this relationship has been generated
715                varyflag = False
716                varname = ''
717                fixedval = None
718            if fixedval is None: # this is a new variable, not a constraint
719                if not varname:
720                    varname = paramPrefix + str(consNum) # no assigned name, create one
721                    consNum += 1
722                mapvar.append(varname)
723                # vary the new relationship if it is a degree of freedom in
724                # a set of contraint equations or if a New Var is flagged to be varied.
725                if VaryFree or varyflag: 
726                    unused = False
727                    varyList.append(varname)
728                    # fix (prevent varying) of all the variables inside the constraint group
729                    # (dependent vars)
730                    for var in varsList:
731                        if var in varyList: varyList.remove(var)
732            else:
733                unused = False
734                mapvar.append(fixedval)
735        if unused: # skip over constraints that don't matter (w/o fixed value or any refined variables)
736            if debug: print('Constraint ignored (all variables unrefined)')
737            if debug: print ('   '+_FormatConstraint(constrDict[rel],fixedList[rel]))
738            continue 
739        dependentParmList.append(varlist)
740        arrayList.append(constrArr)
741        invarrayList.append(np.linalg.inv(constrArr))
742        indParmList.append(mapvar)
743    if msg:
744        print ' *** ERROR in constraint definitions! ***'
745        print msg
746        print VarRemapShow(varyList)
747        raise Exception
748    # setup dictionary containing the fixed values
749    global fixedDict
750    # key is original ascii string, value is float
751    for fixedval in fixedList:
752        if fixedval:
753            fixedDict[fixedval] = float(fixedval)
754
755    if debug: # on debug, show what is parsed & generated, semi-readable
756        print 50*'-'
757        print VarRemapShow(varyList)
758        print 'Varied: ',varyList
759        print 'Not Varied: ',fixedVarList
760
761def StoreEquivalence(independentVar,dependentList):
762    '''Takes a list of dependent parameter(s) and stores their
763    relationship to a single independent parameter (independentVar)
764
765    :param str independentVar: name of master parameter that will be used to determine the value
766      to set the dependent variables
767
768    :param list dependentList: a list of parameters that will set from
769         independentVar. Each item in the list can be a string with the parameter
770         name or a tuple containing a name and multiplier:
771         ``['parm1',('parm2',.5),]``
772
773    '''
774   
775    global dependentParmList,arrayList,invarrayList,indParmList
776    mapList = []
777    multlist = []
778    for var in dependentList:
779        if isinstance(var, basestring):
780            mult = 1.0
781        elif len(var) == 2:
782            var,mult = var
783        else:
784            raise Exception, "Cannot parse "+repr(var) + " as var or (var,multiplier)"
785        mapList.append(var)
786        multlist.append(tuple((mult,)))
787    # added relationships to stored values
788    arrayList.append(None)
789    invarrayList.append(np.array(multlist))
790    indParmList.append(tuple((independentVar,)))
791    dependentParmList.append(mapList)
792    return
793
794def GetDependentVars():
795    '''Return a list of dependent variables: e.g. variables that are
796    constrained in terms of other variables
797
798    :returns: a list of variable names
799
800    '''
801    dependentVars = []
802    global dependentParmList
803    for lst in dependentParmList:
804        for itm in lst: dependentVars.append(itm)
805    return dependentVars
806
807def GetIndependentVars():
808    '''Return a list of independent variables: e.g. variables that are
809    created by constraints of other variables
810
811    :returns: a list of variable names
812
813    '''
814    independentVars = []
815    global indParmList,fixedDict
816    for lst in indParmList:
817        for name in lst:
818            if name in fixedDict: continue
819            independentVars.append(name)
820    return independentVars
821
822def PrintIndependentVars(parmDict,varyList,sigDict,PrintAll=False,pFile=None):
823    '''Print the values and uncertainties on the independent variables'''
824    global dependentParmList,arrayList,invarrayList,indParmList,fixedDict
825    printlist = []
826    mapvars = GetIndependentVars()
827    for i,name in enumerate(mapvars):
828        if name in fixedDict: continue
829        if PrintAll or name in varyList:
830            sig = sigDict.get(name)
831            printlist.append([name,parmDict[name],sig])
832    if len(printlist) == 0: return
833    s1 = ''
834    print >>pFile,130*'-'
835    print >>pFile,"Variables generated by constraints"
836    printlist.append(3*[None])
837    for name,val,esd in printlist:
838        if len(s1) > 120 or name is None:
839            print >>pFile,''
840            print >>pFile,s1
841            print >>pFile,s2
842            print >>pFile,s3
843            s1 = ''
844            if name is None: break
845        if s1 == "":
846            s1 = ' name  :'
847            s2 = ' value :'
848            s3 = ' sig   :'
849        s1 += '%15s' % (name)
850        s2 += '%15.4f' % (val)
851        if esd is None:
852            s3 += '%15s' % ('n/a')
853        else:   
854            s3 += '%15.4f' % (esd)
855
856def ComputeDepESD(covMatrix,varyList,parmDict):
857    '''Compute uncertainties for dependent parameters from independent ones
858    returns a dictionary containing the esd values for dependent parameters
859    '''
860    sigmaDict = {}
861    for varlist,mapvars,invmultarr in zip(dependentParmList,indParmList,invarrayList):
862        #if invmultarr is None: continue # probably not needed
863        valuelist = [parmDict[var] for var in mapvars]
864        # get the v-covar matrix for independent parameters
865        vcov = np.zeros((len(mapvars),len(mapvars)))
866        for i1,name1 in enumerate(mapvars):
867            if name1 not in varyList: continue
868            iv1 = varyList.index(name1)
869            for i2,name2 in enumerate(mapvars):
870                if name2 not in varyList: continue
871                iv2 = varyList.index(name2)
872                vcov[i1][i2] = covMatrix[iv1][iv2]
873        # vec is the vector that multiplies each of the independent values
874        for v,vec in zip(varlist,invmultarr):
875            sigmaDict[v] = np.sqrt(np.inner(vec.T,np.inner(vcov,vec)))
876    return sigmaDict
877
878def _FormatConstraint(RelDict,RelVal):
879    '''Formats a Constraint or Function for use in a convenient way'''
880    linelen = 45
881    s = [""]
882    for var,val in RelDict.items():
883        if var.startswith('_'): continue
884        if len(s[-1]) > linelen: s.append(' ')
885        m = val
886        if s[-1] != "" and m >= 0:
887            s[-1] += ' + '
888        elif s[-1] != "":
889            s[-1] += ' - '
890            m = abs(m)
891        s[-1] += '%.3f*%s '%(m,var)
892    if len(s[-1]) > linelen: s.append(' ')
893    if RelVal is None:
894        s[-1] += ' = New variable'
895    else:
896        s[-1] += ' = ' + RelVal
897    s1 = ''
898    for s2 in s:
899        if s1 != '': s1 += '\n\t'
900        s1 += s2
901    return s1
902
903def VarRemapShow(varyList):
904    '''List out the saved relationships. This should be done after the constraints have been
905    defined using :func:`StoreEquivalence`, :func:`GroupConstraints` and :func:`GenerateConstraints`.
906
907    :returns: a string containing the details of the contraint relationships
908    '''
909    s = ''
910    if len(fixedVarList) > 0:
911        s += 'Fixed Variables:\n'
912        for v in fixedVarList:
913            s += '    ' + v + '\n'
914    s += 'Variable mapping relations:\n'
915    global dependentParmList,arrayList,invarrayList,indParmList,fixedDict
916    for varlist,mapvars,multarr,invmultarr in zip(
917        dependentParmList,indParmList,arrayList,invarrayList):
918        for i,mv in enumerate(mapvars):
919            if multarr is None:
920                s += '  ' + str(mv) + ' is equivalent to parameter(s): '
921                j = 0
922                for v,m in zip(varlist,invmultarr):
923                    #print v,m[0]
924                    if j > 0: s += '  & '
925                    j += 1
926                    s += str(v)
927                    if m != 1:
928                        s += " / " + str(m[0])                       
929                s += '\n'
930                continue
931            s += %s = ' % mv
932            j = 0
933            for m,v in zip(multarr[i,:],varlist):
934                if m == 0: continue
935                if j > 0: s += ' + '
936                j += 1
937                s += '(%s * %s)' % (m,v)
938            if mv in varyList: s += ' VARY'
939            s += '\n'
940    s += 'Inverse variable mapping relations:\n'
941    for varlist,mapvars,invmultarr in zip(dependentParmList,indParmList,invarrayList):
942        for i,mv in enumerate(varlist):
943            s += %s = ' % mv
944            j = 0
945            for m,v in zip(invmultarr[i,:],mapvars):
946                if m == 0: continue
947                if j > 0: s += ' + '
948                j += 1
949                s += '(%s * %s)' % (m,v)
950            s += '\n'
951    return s
952
953def Dict2Deriv(varyList,derivDict,dMdv):
954    '''Compute derivatives for Independent Parameters from the
955    derivatives for the original parameters
956
957    :param list varyList: a list of parameters names that will be varied
958
959    :param dict derivDict: a dict containing derivatives for parameter values keyed by the
960      parameter names.
961
962    :param list dMdv: a Jacobian, as a list of np.array containing derivatives for dependent
963      parameter computed from derivDict
964
965    '''
966    global dependentParmList,arrayList,invarrayList,indParmList,invarrayList
967    for varlist,mapvars,multarr,invmultarr in zip(dependentParmList,indParmList,arrayList,invarrayList):
968        for i,name in enumerate(mapvars):
969            # grouped variables: need to add in the derv. w/r
970            # dependent variables to the independent ones
971            if name not in varyList: continue # skip if independent var not varied
972            if multarr is None:
973                for v,m in zip(varlist,invmultarr):
974                    #print 'add derv',v,'/',m[0],'to derv',name
975                    if m == 0: continue
976                    dMdv[varyList.index(name)] += derivDict[v] / m[0]
977            else:
978                for v,m in zip(varlist,multarr[i,:]):
979                    #print 'add derv',v,'*',m,'to derv',name
980                    if m == 0: continue
981                    dMdv[varyList.index(name)] += m * derivDict[v]
982
983def Map2Dict(parmDict,varyList):
984    '''Create (or update) the Independent Parameters from the original
985    set of Parameters
986
987    Removes dependent variables from the varyList
988
989    This should be done once, after the constraints have been
990    defined using :func:`StoreEquivalence`,
991    :func:`GroupConstraints` and :func:`GenerateConstraints` and
992    before any variable refinement is done
993    to complete the parameter dictionary by defining independent
994    parameters and satisfying the constraint equations.
995
996    :param dict parmDict: a dict containing parameter values keyed by the
997      parameter names.
998      This will contain updated values for both dependent and independent
999      parameters after Dict2Map is called. It will also contain some
1000      unexpected entries of every constant value {'0':0.0} & {'1.0':1.0},
1001      which do not cause any problems.
1002
1003    :param list varyList: a list of parameters names that will be varied
1004   
1005
1006    '''
1007    # process the Independent vars: remove dependent ones from varylist
1008    # and then compute values for the Independent ones from their dependents
1009    global dependentParmList,arrayList,invarrayList,indParmList,fixedDict
1010    for varlist,mapvars,multarr in zip(dependentParmList,indParmList,arrayList):
1011        for item in varlist:
1012            try:
1013                varyList.remove(item)
1014            except ValueError:
1015                pass
1016        if multarr is None: continue
1017        valuelist = [parmDict[var] for var in varlist]
1018        parmDict.update(zip(mapvars,
1019                            np.dot(multarr,np.array(valuelist)))
1020                        )
1021    # now remove fixed variables from the varyList
1022    global fixedVarList
1023    for item in fixedVarList:
1024        try:
1025            varyList.remove(item)
1026        except ValueError:
1027            pass
1028    # Set constrained parameters to their fixed values
1029    parmDict.update(fixedDict)
1030
1031def Dict2Map(parmDict,varyList):
1032    '''Applies the constraints defined using :func:`StoreEquivalence`,
1033    :func:`GroupConstraints` and :func:`GenerateConstraints` by changing
1034    values in a dict containing the parameters. This should be
1035    done before the parameters are used for any computations
1036
1037    :param dict parmDict: a dict containing parameter values keyed by the
1038      parameter names.
1039      This will contain updated values for both dependent and independent
1040      parameters after Dict2Map is called. It will also contain some
1041      unexpected entries of every constant value {'0':0.0} & {'1.0':1.0},
1042      which do not cause any problems.
1043
1044    :param list varyList: a list of parameters names that will be varied
1045   
1046    '''
1047    global dependentParmList,arrayList,invarrayList,indParmList,fixedDict
1048    # reset fixed values (should not be needed, but very quick)
1049    # - this seems to update parmDict with {'0':0.0} & {'1.0':1.0} - probably not what was intended
1050    # not needed, but also not a problem - BHT
1051    parmDict.update(fixedDict)
1052    for varlist,mapvars,invmultarr in zip(dependentParmList,indParmList,invarrayList):
1053        #if invmultarr is None: continue
1054        valuelist = [parmDict[var] for var in mapvars]
1055        parmDict.update(zip(varlist,
1056                            np.dot(invmultarr,np.array(valuelist)))
1057                        )
1058
1059#======================================================================
1060# internal routines follow (these routines are unlikely to be called
1061# from outside the module)
1062
1063def GramSchmidtOrtho(a,nkeep=0):
1064    '''Use the Gram-Schmidt process (http://en.wikipedia.org/wiki/Gram-Schmidt) to
1065    find orthonormal unit vectors relative to first row.
1066
1067    If nkeep is non-zero, the first nkeep rows in the array are not changed
1068   
1069    input:
1070       arrayin: a 2-D non-singular square array
1071    returns:
1072       a orthonormal set of unit vectors as a square array
1073    '''
1074    def proj(a,b):
1075        'Projection operator'
1076        return a*(np.dot(a,b)/np.dot(a,a))
1077    for j in range(nkeep,len(a)):
1078        for i in range(j):
1079            a[j] -= proj(a[i],a[j])
1080        if np.allclose(np.linalg.norm(a[j]),0.0):
1081            raise Exception,"Singular input to GramSchmidtOrtho"
1082        a[j] /= np.linalg.norm(a[j])
1083    return a
1084
1085def _FillArray(sel,dict,collist,FillDiagonals=False):
1086    '''Construct a n by n matrix (n = len(collist)
1087    filling in the rows using the relationships defined
1088    in the dictionaries selected by sel
1089
1090    If FillDiagonals is True, diagonal elements in the
1091    array are set to 1.0
1092    '''
1093    n = len(collist)
1094    if FillDiagonals:
1095        arr = np.eye(n)
1096    else:
1097        arr = np.zeros(2*[n,])
1098    # fill the top rows
1099    for i,cnum in enumerate(sel):
1100        for j,var in enumerate(collist):
1101            arr[i,j] = dict[cnum].get(var,0)
1102    return arr
1103
1104def _SwapColumns(i,m,v):
1105    '''Swap columns in matrix m as well as the labels in v
1106    so that element (i,i) is replaced by the first non-zero element in row i after that element
1107
1108    Throws an exception if there are no non-zero elements in that row
1109    '''
1110    for j in range(i+1,len(v)):
1111        if not np.allclose(m[i,j],0):
1112            m[:,(i,j)] = m[:,(j,i)]
1113            v[i],v[j] = v[j],v[i]
1114            return
1115    else:
1116        raise Exception,'Singular input'
1117
1118def _RowEchelon(m,arr,collist):
1119    '''Convert the first m rows in Matrix arr to row-echelon form
1120    exchanging columns in the matrix and collist as needed.
1121
1122    throws an exception if the matrix is singular because
1123    the first m rows are not linearly independent
1124    '''
1125    n = len(collist)
1126    for i in range(m):
1127        if np.allclose(arr[i,i],0):
1128            _SwapColumns(i,arr,collist)
1129        arr[i,:] /= arr[i,i] # normalize row
1130        # subtract current row from subsequent rows to set values to left of diagonal to 0
1131        for j in range(i+1,m):
1132            arr[j,:] -= arr[i,:] * arr[j,i]
1133
1134if __name__ == "__main__":
1135    parmdict = {}
1136    constrDict = [
1137        {'0:12:Scale': 2.0, '0:11:Scale': 1.0, '0:14:Scale': 4.0, '0:13:Scale': 3.0, '0:0:Scale': 0.5},
1138        {'0:0:eA': 0.0},
1139        {'2::C(10,6,1)': 1.0, '1::C(10,6,1)': 1.0},
1140        {'1::C(10,0,1)': 1.0, '2::C(10,0,1)': 1.0},
1141        {'1::AUiso:0': 1.0, '0::AUiso:0': 1.0},
1142        {'0::A0': 0.0}
1143        ]
1144    fixedList = ['5.0', '0', None, None, '1.0', '0']
1145    StoreEquivalence('2::atomx:3',('2::atomy:3', ('2::atomz:3',2,), ))
1146    #StoreEquivalence('1::atomx:3',('2::atomx:3', ('2::atomz:3',2,), )) # error: dependent & independent vars mixed
1147    #StoreEquivalence('1::atomx:3',('2::atomy:3', ('2::atomz:3',2,), )) # error: dependent vars repeated
1148    #StoreEquivalence('0:1:eA',('0:0:eA',)) # error: equiv & fixed
1149    #StoreEquivalence('0:99:Scale',('0:12:Scale',)) # error: equiv & constrained
1150    #StoreEquivalence('0:12:Scale',('0:99:Scale',)) # error: equiv & constrained
1151    varylist = ['2::atomx:3',
1152                '2::C(10,6,1)', '1::C(10,6,1)',
1153                '2::atomy:3', '2::atomz:3',
1154                '0:12:Scale', '0:11:Scale', '0:14:Scale', '0:13:Scale', '0:0:Scale']
1155#    e,w = CheckConstraints([,
1156#                     [{'2:0:Scale': 1.0, '5:0:Scale': 1.0, '10:0:Scale': 1.0, '6:0:Scale': 1.0, '9:0:Scale': 1.0, '8:0:Scale': 1.0,# '3:0:Scale': 1.0, '4:0:Scale': 1.0, '7:0:Scale': 1.0, '1:0:Scale': 1.0, '0:0:Scale': 1.0}],
1157#                     ['1.0'])
1158#    if e: print 'error=',e
1159#    if w: print 'error=',w
1160#    varyList = ['0::A0', '0::AUiso:0', '0::Afrac:1', '0::Afrac:2', '0::Afrac:3', '0::Afrac:4', '0::dAx:5', '0::dAy:5', '0::dAz:5', '0::AUiso:5', ':0:Back:0', ':0:Back:1', ':0:Back:2', ':0:Back:3', ':0:Back:4', ':0:Back:5', ':0:Back:6', ':0:Back:7', ':0:Back:8', ':0:Back:9', ':0:Back:10', ':0:Back:11', ':0:U', ':0:V', ':0:W', ':0:X', ':0:Y', ':0:Scale', ':0:DisplaceX', ':0:DisplaceY']
1161#    constrDict = [
1162#        {'0::Afrac:4': 24.0, '0::Afrac:1': 16.0, '0::Afrac:3': 24.0, '0::Afrac:2': 16.0},
1163#        {'0::Afrac:1': 1.0, '0::Afrac:2': 1.0},
1164#        {'0::Afrac:4': 1.0, '0::Afrac:3': 1.0}]
1165#    fixedList = ['40.0', '1.0', '1.0']
1166
1167    errmsg, warnmsg = CheckConstraints(varylist,constrDict,fixedList)
1168    if errmsg:
1169        print "*** Error ********************"
1170        print errmsg
1171    if warnmsg:
1172        print "*** Warning ********************"
1173        print warnmsg
1174    if errmsg or warnmsg:
1175        sys.exit()
1176    groups,parmlist = GroupConstraints(constrDict)
1177    GenerateConstraints(groups,parmlist,varylist,constrDict,fixedList)
1178    print VarRemapShow(varylist)
1179    parmdict.update( {
1180        '0:12:Scale': 1.0, '0:11:Scale': 1.0, '0:14:Scale': 1.0, '0:13:Scale': 1.0, '0:0:Scale': 2.0,
1181        '0:0:eA': 0.0,
1182        '2::C(10,6,1)': 0.2, '1::C(10,6,1)': 0.3,
1183        '1::C(10,0,1)': 0.2, '2::C(10,0,1)': 0.3,
1184        '1::AUiso:0': 0.02, '0::AUiso:0': 0.03,
1185        '0::A0': 0.0,
1186        '2::atomx:3':0.23,'2::atomy:3':-.23, '2::atomz:3':-0.11,
1187        })
1188    print 'parmdict start',parmdict
1189    print 'varylist start',varylist
1190    before = parmdict.copy()
1191    Map2Dict(parmdict,varylist)
1192    print 'parmdict before and after Map2Dict'
1193    print '  key / before / after'
1194    for key in sorted(parmdict.keys()):
1195        print '  '+key,'\t',before.get(key),'\t',parmdict[key]
1196    print 'varylist after',varylist
1197    before = parmdict.copy()
1198    Dict2Map(parmdict,varylist)
1199    print 'after Dict2Map'
1200    print '  key / before / after'
1201    for key in sorted(parmdict.keys()):
1202        print '  '+key,'\t',before.get(key),'\t',parmdict[key]
1203#    dMdv = len(varylist)*[0]
1204#    deriv = {}
1205#    for i,v in enumerate(parmdict.keys()): deriv[v]=i
1206#    Dict2Deriv(varylist,deriv,dMdv)
Note: See TracBrowser for help on using the repository browser.