Sequential refinement of multiple datasets – CuCr2O4 from 7K to 300K


Sequential refinement is a way to fit a series of datasets with a set of closely related models that evolve over the course of the measurements.  Sequential refinements can be performed in two different ways:

A: Where the resulting parameters after refinement with a histogram are used as the starting point for the next histogram.
When this mode is used, the same parameters must be varied in each histogram (phase-based parameters, including those in the phase/data tab are forced to be the same, since there is only one set of these.) This is commonly used for an initial refinement.
B: Where parameters are not copied between histograms.
When this is done, the phase-based parameters from refinement with each histogram are still used as the starting point for the next, but histogram parameters may start all at the same place or may be started at different values, for example to continue a previous sequential refinement to convergence or with more parameters. Note that in this case different sets of parameters may be refined for each histogram.

In this example, we will fit a temperature series. Note that it is also possible to fit a temperature series as a combined refinement and, as will be seen, one can still deal with thermal expansion (using the Hydrostatic/Elastic strain terms), but a combined refinement would average the atomic parameters across all temperatures. A sequential refinement allows each histogram to have differing atomic parameters; their variation with temperature might be of interest. Likewise, the parameters on the phase/data tab, such as size/microstrain coefficients, phase fractions, are also allowed to be different for each histogram, if varied.


This example assumes that one is already familiar with use of GSAS-II for simple refinements and have started GSAS-II without a project.

Note that most of the screen images shown in this tutorial are taken from GSAS-II running on a Mac OS X machine; these illustrate that the menus are not located on the data tree or data editing windows.

If you download this tutorial, data files will be placed on your computer and the open file dialogs should bring you to that location. If not, the files can be downloaded from here.

Step 1: Fit a single temperature.


1.      Use the Import/Phase/from CIF file menu item to read the phase information


2.      In the file browser dialog window that is created, change the file type fromCIF filetozip archive and select the file (change the file directory to SeqRefine/data). Another dialog window will be created showing all the files in the zip archive. To simplify the view to only show the phase files, type “cif” (or “CIF” – case is ignored) into the filter. Select the CuCr2O4.cif and the CuO.cif input files. Press OK.

(Note that if files CuCr2O4.cif or the CuO.cif already exist, you will be asked to OK overwriting them.)


3.      You will be asked sequentially for the name for each phase. The defaults (CuCr2O4 and CuO) are fine, so OK can be pressed for each question. 


4.       Read in the ~7 K dataset (OH_00.fxye) using Import/Powder Data/from GSAS powder data file to open a file browser. Again selectzip archiveand the file. Select file OH_00.fxye (use of the filter can again be helpful) and press OK.

You will be shown the first few lines from the file. Confirm that the selected file is correct by pressing OK.

Note that the instrument parameter file with the matching name (OH_00.prm) is read automatically.


5.      Map the phases to the data: After reading in the dataset, a window used to select phases is shown. Select both phases and press OK. 


6.      We want to fit the lattice parameters, background, and a phase fraction initially.  To illustrate how constraints are used, we will refine both phase fractions, but constrain their sum to be one. (A simpler alternate approach would be to refine only a single phase fraction.) To set up the constraint, click on the Constraints item in the data tree. Then click on the Histogram/Phase tab in the data editing window. Then in the Edit Constr. menu  click on the Edit Constr/Add constraint equation menu item.


In the next window use a Filter of scale to make the list shorter and then select the phase fraction for the first phase 0:0:Scale (the second one, 1:0:Scale, would work too)


In the next window, either select the other phase fraction (1:0:Scale) or select all phases (all:0:Scale)

This creates the desired constraint, which forces the sum to be 1. (If the desired sum was different or different multipliers were desired, press the Edit button for the constraint.)


7.      Now we select the variables to refine.


Select the first phase (CuCr2O4) , which should open the General Tab. Click on the Refine unit cell option (If the phases are not shown, click on the triangle to the left of the Phase entry to “open” the tree item). We also want to refine the phase fraction for this phase, so select the Data tab and click on the Phase fraction box to cause that to be refined. 

Repeat this with the second phase (CuO), by selecting it in the data tree. Then select the General tab and on the Refine unit cell option. We also want to refine the phase fraction for this phase, so select the Data tab and click on the Phase fraction box to cause that to be refined.


Also select the Background entry under PWDR OH_00.fxye Bank 1 in the GSAS-II data tree (again if this is not shown, click on the triangle to the left of the histogram entry to “open” the tree item.) Confirm that Background function should already be set as chebyshev and the Refine flag should be checked. Change the No. coeff. to 6 to introduce more background terms.


The histogram scale factor, on the Sample Parameters entry should already be set as refined by default, but you may want to confirm this.


Finally, note that the first broad peak in these data is spurious. It is known to arise from the kapton window in the cryostat used for sample cooling; this was confirmed by collecting data without a sample. We could model this peak, but it is easier to simply drop the data below the first peak at ~5 degrees. To do this, select the Limits entry for histogram PWDR OH_00.fxye Bank 1 and in the New Tmin box (to lower left) change it to 4.5.


8. Optional: confirm that the initial parameters have been selected correctly for refinement using the Calculate/View LS parms menu item for the main GSAS-II data ttree window. There will be 15 refined parameters: 0::Ax (x=0,1,2) and 1::Ax (x=0,1,2,4), the reciprocal lattice metric terms for each phase; :0:Scale, the histogram scale factor, :0:Back:x (x=0-5), the six background coefficients and ::constr0 which is generated from the constraints. The phase factors, x:0:Scale (x=1,2) are shown as constrained.


9.      Use the Calculate/Refine main GSAS-II menu item to initiate the refinement. Since the project has not been saved, you will be asked to select a name (and optionally a directory) for the GSAS-II project (.gpx) file.  The file name SeqTut.gpx is assumed here. The refinement progresses quickly to a wR of ~17%.


10.  There are several additional variables that should now be added. One should always refine a sample displacement parameter or a zero correction. In most cases with Debye-Scherrer geometry, the best choice is to add the Sample X displ. parameter on the Sample Parameters section of histogram PWDR OH_00.fxye Bank 1. (NB: a data range >>90 degrees is needed before the Sample Y displ. term can be included). Note that since this instrument is very well characterized, there is no need to refine any of the instrumental terms.


Selecting the PWDR OH_00.fxye Bank 1 entry shows the “Rietveld plot” and zooming in on different regions of the pattern provides insight on how the fit could be improved. You can alter the view to show the square root of the intensity (use the ‘S’ key to toggle this). The background is not well fit in the low angle range and examination shows a “hump” of low-angle scattering from ~4.5 to 6 degrees. This is more easily fit by adding a background peak than extra terms to the background function. Select the Background item in histogram PWDR OH_00.fxye Bank 1 and change Peaks in Background to 1.  Change the peak position (pos) to the approximate peak center (5.25 degrees) and set the intensity (int) and Gaussian width (sig) to be refined.


Once the fit has been improved we can then add the position of this peak to the fit.


While in other projects I might postpone this until later, we can also start on improving the peak fit. We will do this by varying the sample broadening for the major phase (since the instrumental terms are very good.) Select the first phase (CuCr2O4) and its Data tab, click on the size and microstrain parameters to add them to the fit. 


There are now 20 refined parameters and a repeat of the fitting (Use the Calculate/Refine menu item) causes the fit to improve to a wR of <16%.


11.  We can now include the position flag (pos/refine) for the background peak in the refinement. Do this at the Background item, as before.


Close examination of the pattern shows that the peak widths are still not as well fit as desired. Since strain broadening seems more significant that sample broadening, we could use an anisotropic strain model. Rather than choosing the relatively complex generalized model, we can introduce only two terms with the uniaxial model.  Trial and error will show that the 010 direction works best. Click on the first phase (CuCr2O4) and Data tab and select uniaxial for the Mustrain model and use 0 1 0 for the Unique axis, HKL . Select refinement for both Equatorial and Axial mustrain.


Likewise, introduce a peak broadening parameter for the second phase by clicking on CuO in the data tree and click on the single microstrain (isotropic) term for this second phase.


12. Repeating the refinement causes the wR to drop to ~14%. Clicking on the Covariance data tree item shows that the goodness of fit (GOF) is around 1.1 (corresponding to a reduced χ2 <1.2) with 23 refined parameters. While atomic parameters could now be refined, we are already have a good enough fit to start a refinement with all the datasets, as the plot clearly shows.

Step 2: Prepare for sequential fit.


There is one significant difference between fitting a sequence of datasets versus fitting a single data set. This is in the way that lattice constants are fit. In GSAS-II the lattice constants for each phase are determined from reciprocal metric tensor terms, (which are labeled <p>::Axx where <p> is phase number and the xx value(s) depend on the unit cell symmetry) which are added to Hydrostatic/Elastic strain terms (labeled <p>:<h>:Dxx, as above but <h> is the histogram number). Note that there is only one set of <p>::Axx terms for each phase, but there is a set of <p>:<h>:Dxx for each phase and histogram. Since in most cases in a sequential refinement, one wants to have differing lattice constants for each histogram, refinement of the <p>::Axx terms in a sequential refinement is reported as an error.


Note that the <p>::Axx are controlled by the "Refine unit cell" option on the Phase/General tab, while the <p>:<h>:Dxx are listed on the Phase/Data tab, along with all the other parameters where there are values for each histogram and phase. Note that in a sequential fit, there is only one set of l (for which there will be only one set for each phase in the entire refinement), we will use the Hydrostatic/Elastic strain terms (parameters labeled <p>:<h>:Dxx where <p> is phase number and <h> is histogram number and the xxvalue(s) depend on the unit cell symmetry) to treat the change in lattice parameters. These items describe the change in lattice parameters and appear in the Phase Data tab, where there is a set for each histogram assigned to the phase.


We will make a switch from refining <p>::Axx terms to <p>:<h>:Dxx terms for the current dataset, in preparation for the sequential refinement with all datasets. Also at this point we will temporarily remove variables that could make the fit diverge as the lattice constants shift. As an example, when initially fitting a model with peaks that overlap with the observed peaks, but not very well, the fit might be improved by a change in the phase fractions (which might eliminate our impurity CuO phase) or make peaks quite a bit broader than appropriate, these changes could result in a model that is less correct than the starting point but still be a local LS minimum. After such a change the best-fitting model may never be found. Accordingly, we will turn off refinement of all parameters except those that determine peak positions (the Dxx terms) and the overall scale factor. We will also include refinement of the background terms since they are unlikely to affect convergence.


1.      Select the first phase (CuCr2O4) and on the General tab, click on the Refine unit cell option to turn off refinement.



2.      On the Data tab, click on the D11, D22 and D33 terms to refine the lattice strain terms. While here, to simplify convergence for the sequential refinement, where lattice constants change significantly, turn off the refinement of the three peak width terms and the phase fraction. The Data window will appear similar to what is below:


3.      Click on the second phase (CuO) and on the General tab, click on the Refine unit cell option to turn off refinement.


On the Data tab for CuO, click on the D11, D22, and D33 terms to refine the lattice strain terms and turn off refinement of the phase fraction.  We could refine the D13 term (which corresponds to the CuO β* angle). However, due to the small number of the small number of CuO peaks that are in the pattern, the refinement to be relatively insensitive to this parameter and convergence will be more rapid if this is added to the refinement later.


4.      To also simplify the refinement at this stage, turn off the all the background refinement terms by clicking on Background on the histogram tree item and removing checks the four items that are refined (overall terms and peak intensity, position and sig.)


Likewise, for the histogram Sample Parameters, turn off the Sample X displ. parameter by clicking on the Sample Parameters item in the histogram entries.


Optional: One can confirm that the number of parameters is now 7 (3 Dxx terms for phase 1, 3 Dxx terms for phase 2 and a scale factor.) Also, while it is not necessary, as a test it is possible to repeat the refinement. If this is done, the Dxx values will refine to values very close to zero (within the refinement uncertainties) indicating that the lattice parameters were already optimal and the quality of the fit (noted by wR or GOF) will be largely unchanged.

Step 3: Import the remaining datasets histograms.


In this step we bring in the remaining data and ensure that the two phases are linked to each new histogram.


1.      The remaining datasets are read using Import/Powder Data/from GSAS powder data file to open a file browser. Again select zip archive and the downloaded file. Select all the .fxye files except OH_00.fxye (use of the filter can again be helpful to show just the data file; press Set All and then unselect the one unneeded file).


This time one must select the instrument parameter file.


Set all data sets to use the two phases, as before in step 1 #5.


Note that the files must be read in in the sequence they will be processed. This is not an issue here, but for some sequential refinements, one may need to use multiple Import operations to read the files in the desired order or rename files so they are listed in the correct order in the file browser.


Optional: Note that the default graphics mode is to display the last dataset read in.  If you want to see the entire ensemble of data sets, click on the main entry for any of the datasets (for example PWDR OH_00.fxye Bank 1), click on the frame of the plot window and press the “m” key.  (Alternately, press the “K” icon from the list at the bottom of plot, between the “floppy disk” and “question mark” icons and then select “m: toggle multidata plot” from the pop-up window). The ‘u’, ‘d’, ‘r’ and ‘l’ keys control the appearance of a ‘‘waterfall plot’’. You can also use the ‘c’ key to obtain a contour plot of all the data, as below; in this mode the ‘u’ and ‘d’ keys change the upper level on the color scale, changing the contrast of the plot.

Above, I have used the ‘d’ key to increase the contrast & zoomed in to a an angular location that shows the effect of temperature on the peak positions.

Step 4: Duplicate parameters across histograms and phase/histogram entries.


In this step we make sure that the starting parameters are the same for all histogram as well as the phase/histogram parameters (in the Phase/Data tab).


1. Copy the histogram parameters to be the same in all histograms: In this case we have changed limits and varied background and sample parameter values in the first histogram. We will copy those values to all histograms. Select the original histogram, entry PWDR OH_00.fxye Bank 1 and then use the menu command Commands/Copy params.


This causes a dialog box where the different histogram parameters sets to be copied are selected. We need to duplicate the Limits, Background and Sample Parameters between all histograms. (Note that the Instrument Parameters have not been changed in this exercise from what was read from the same instrument parameter file so they are already the same, but there is no harm in copying them, so the default entries do not need to be changed.) Press OK. Note that this action will only copy the initial values the Sample Parameters section (including all that can be refined) but will not change values such as as setting angles, temperature, etc.


An additional dialog window is shown which asks which histograms should receive the parameters. Press Set All to select all remaining histograms and then press OK.


It can be confirmed that the copy has been performed by examining any of the additional histogram plots (to see the changed lower limit) or looking at Background or Sample Parameter values in the Data Editing window.


2. Next we need to copy the Phase/Histogram parameters (in Phase/Data tab) for each phase: Select the first phase (CuCr2O4) and click on its Data tab. When this tab is first pressed, the first histogram linked to this phase (first entry in the list at the upper left) is automatically displayed and the parameters associated with this phase and histogram are displayed. These need to be copied to every other histogram. To do this select the Copy data menu item from the Edit Phase menu. A dialog window then appears to select where to copy the parameters. Select all histograms by pressing the Set All button. Then press the OK button.



It can be verified that the copy took place by clicking on different histograms in the list on the upper left of the Data tab window.


3.      Repeat this copying the Phase/Histogram parameters within the second phase by clicking on it (CuO) in the data tree (and possibly on the Data tab.) Then click on the Edit Phase/Copy data menu item and again select all histograms and press OK as in the last step.


Step 5: Setup and start the sequential refinement.


At this point we set which histograms will be used in the fit and set option that the resulting parameters from the results of each refinement will be used for the next refinement.


1.      First, select the Controls item in the data tree and press the Select data button.

This opens a window where all histograms can be selected (it would be equally valid to skip the first histogram, since that has already been fully fit). Press OK and note that the Controls window now lists the number of datasets in the sequential refinement and two new controls.


2.      Check the Copy results to next histogram checkbox. This is essential. If this is not selected, then every refinement will start with the current parameters (which are all identical, since they were set with the Copy operations.) The first few refinements will go well, but after the temperature has changed enough, the starting lattice parameters (determined from the cell and with Dxx, where the latter values are near zero) will be too far off to align with the peaks and the refinements will fail.


Note that the Reverse order checkbox causes the sequential refinement to start with the last histogram, not the first. Sometimes this is useful, but not here.


3.      Change Max cycles to 10. By increasing the number of cycles, we allow the lattice parameters to to fully adjust to the current histogram.


4.      Start the sequential refinement with Calculate/Sequential refine; this will take some time as the refinement works its way through all the histograms.


Note that if the copy operations in Step 4 were not performed correctly, the refined parameters in each refinement will not be the same and the refinement will stop with an error such as this:

Text Box: **** ERROR - variable list for this histogram does not match previous

current histogram PWDR OH_04.fxye Bank 1 has 6 variables
Variables in previous but not in current: 1::D22, 1::D33, 1::D11, 

Previous refinement has 7 variables
Variables in current but not in previous: 1::Scale, 1::Mustrain;i,

(This error results if the parameters for the second phase were not copied in part 3 so that the phase fraction and strain broadening are retained for all but the first histogram and the 1::Dii terms are only included the first.)

Step 6: Examine the sequential refinement results.


1.      When the refinement completes, a window will be displayed where you are asked if you want to load the results:


After doing OK, the Sequential Results item in the data tree is automatically selected, which displays a summary of the refinement results as a large table.



The second column shows the weighted profile R-factor, from each fit. This shows a steady increase with temperature, as might be expected if there were a systematic change in some sample parameters, such as strain or background or even atomic coordinates (all of which have not yet been individually fit). The Δχ2 value in the third column shows the percent improvement in χ2 seen in the last cycle of refinement. (As an example, if χ2 were to change from 4.1 to 4.0, then the value will be 100*(4.1-4.0)/4.0, or 2.5). Since the default convergence criterion in the controls is 0.0001 (or 0.01%) small values, as shown above, indicate the refinement converged. It is good idea to make sure that shifts are small, if need be by running further cycles of refinement, before adding more parameters to be varied.


Subsequent columns will show any of the sample parameters that vary across the datasets, here only temperature and then the direct cell lattice constants for each phase, generated from the Ax and Dij terms. Scrolling further shows the values of each refined parameter.


Before going farther, it is useful to review the trends in the lattice parameters as a sanity check. This can be done by double-clicking on a column of the table, such as the 0::b parameter, which displays this plot:



Pressing the “K” icon and selecting the "s - select x-axis" option or simply pressing the “s” key brings up a menu where the x-axis can be selected as temperature. (If the plot fails to update, click on another column and then return to the original column.)



Step 7: Continue the sequential refinement.


At this point we choose to continue the refinements from their current values (note that the Copy results to next histogram checkbox is automatically unchecked after a successful sequential refinement.) including some additional parameters in the refinement. We will do this is a few steps.


1.      Restore the refinement of background parameters as before. Do this by selecting the Background tree item for any one histogram, set the flags, as below.



Then use the File/Copy flags menu item to copy these refinement flags (only) to all other histograms. NB: the File/Copy menu item would copy the parameter values as well the flags; in this case it would not matter, since all the background values are still the same, but in general this is probably not what you will want to do here.



2.      Likewise, resume varying the Sample X displacement parameter on the Sample Parameters tab for any one histogram and again use the Command/Copy flags menu item to all other histograms.


Description: BHT3 HD:private:tmp:s1.png


3.      For the Phase/histogram parameters add the three broadening terms for the CuCr2O4 (CuCr2O4; data tab) phase; then use the Edit Phase/Copy flags menu item so that they are varied for all histograms. Do not use "Edit Phase/Copy data" menu item, as this would reset all the lattice constant displacements to a single set of values, wiping out the results of the previous refinement.



For the CuO phase, add the single sample broadening term and the remaining lattice parameter via the D13 term.  Again, use the Copy flags? button.


Note that we are not varying the Phase fractions, as this will require a change in the constraints, which will be discussed later.


4.      Start the sequential refinement again with the Calculate/Sequential refine menu item. The fits are significantly improved at higher temperatures, with Rwp values between 13.8 and 17.

Step 8: Sequential refinement constraints


1. We can now add the phase fractions into the refinement, by clicking on the Phase fraction checkbox and Copy Flags button for both phases. However, if the refinement is attempted at this point a subtle error will occur. If one looks at the phase fractions for all but the first histogram, the sum of the phase fractions will not add up to one. The reason for this is because the constraint we added in Step 1, section 6. only constrained the values of 0:0:Scale and 1:0:Scale.


We could define a new constraint for each histogram (on 0:i:Scale and 1:i:Scale for i=1,2,...), but this would be rather tedious. We have an alternate way to define this using a single constraint where a wildcard (*) is used for the histogram number. Wildcards only apply to sequential fits and causes the current histogram number to be inserted for the * when each histogram is used.


To set up the new constraint, click on the Constraints item in the data tree and the Select tab/Histogram/Phase menu item, and then click on the Edit Constr./Add constraint equation menu item. From the menu, we want one of the two x:*:Scale wild card phase fractions; they are at the bottom of the list


There will only be one possible matching variable in the next window, so select that and press OK.


The initial and the most recently defined constraints will then be shown. These two constraints do the same thing for the first histogram and do not interfere but it is best to remove the orignal constraint for simplicity.


To remove the initial constrain press the Delete button in front of the first constraint to remove it.



2. Start the sequential refinement again with the Calculate/Sequential refine menu item. Three new columns appear at the end of the table for ::constr0 (generated by the constraint) and the initial variables, 0:*:Scale and 1:*:Scale. Changes to the Rwp values are quite small, even though the phase fraction for the minor phase changes significantly. This indicates that the refinement is relatively insensitive to these values. This would be a good argument for fixing the value as a constant.

Step 9: Atomic parameter refinement


The last thing we will add to the refinement will be atomic parameters. Note that these are handled a bit differently than the previous things we have refined. There is only one set of coordinates for each phase and these will be refined for each histogram, but are not saved individually, except in the summary table.


This means that after the sequential refinement has been run, the atomic parameters in the phase(s) will correspond to the last histogram. If we repeat the sequential refinement, the coordinates, etc. from the last histogram will be used to fit the first histogram. This may work fine, but if the values have changed significantly between the first and last histogram, there could be problems. The Reverse order checkbox on the Controls causes the sequential refinement to start with the last histogram, which would avoid this issue.


1. Select the CuCr2O4 phase to and then the Atoms tab. Double-click on the refine label at the top of the atoms table, a refinement controls dialog opens, select X to refine coordinates (only O has coordinates not fixed by symmetry) and U to refine the atomic displacement (Uiso) values.  Press OK.


Note the change in the atom table:

Note that since these parameters are shared directly in all histograms, no Copy or Copy flags menu item is needed or even available.


2. Start the sequential refinement again with the Calculate/Sequential refine menu item. The fits are somewhat at the higher temperatures, with Rwp values between 13.8 and 16.4. As would be expected the Uiso values increase with temperature, but the Uiso values for the O atom are less than 0.0 at the lowest temperature. This indicates that these data are insufficient to determine atomic displacements accurately for such a relatively light atom.

This completes the sequential refinement tutorial, although several more things could be attempted to further improve the refinement quality. Be sure to save the project file by selecting the File/Save project menu item, since this will used in a later tutorial. In the Parametric Fitting and Pseudo Variables for Sequential Fits tutorial, functions of fitted parameters are computed and plotted and results are fitted to equations of state.

Last tested with GSAS-II version 2841
Last modified: Fri May 12 18:18:37 CDT 2017