wiki:G2onPy3

Context Navigation

Version 9 (modified by toby, 2 years ago) (diff)
--

GSAS-II with Python 3.x

*This page is out of date but is being retained to keep build information at bottom*

GSAS-II has been packaged with Python 3.6, wxPython 4.0 and the lastest compatible versions of all other packages into a single-download installer. While a single set of files is used to run GSAS-II, regardless of platform or Python/package versions, there are potential differences in how some code behaves with these changes. We consider these packages to be somewhat experimental until we gain more experience and would encourage experienced users to try them out, before removing their existing Python 2.7-based GSAS-II installation.

Windows

If you already have GSAS-II installed and want to keep it, rename the desktop shortcut before starting. Alternately, the previous install directory (usually named ...\g2conda\ can be deleted.)

Note: See "Installing GSAS-II with a single download" instructions for greater detail on these steps, but in them replace the referenced .exe file with one below.

Download:

Note that either version will run on 64-bit windows, but the 64-bit version is preferred.

Run the downloaded file
Use the Install for "Just Me" option
Install in a new directory and avoid paths that contain spaces
Enter a blank line for the proxy, unless your site requires one (not common)

Linux (64-bit only)

Note: See "Installing GSAS-II for Linux with a single download" instructions for greater detail on these steps, but note the referenced file should be replaced with g2conda-1.0.1-Linux-x86_64.sh.

Download: 64-bit Linux: https://anl.box.com/v/g2conda101-linux64

Execute the downloaded file:

    bash <path>/g2conda-1.0.1-Linux-x86_64.sh

Press enter to read through the copyright notice and type "yes" to accept it.
Enter a location for where the files should be installed, if the default is not OK.
- Enter a blank line for the proxy, unless your site requires one (not common)

You will run GSAS-II with a command-line command such as

~/g2conda/bin/python ~/g2conda/GSASII/GSASII.py

where ~/g2conda/ is replaced with your install location.

Mac (OS X)

Note: See "Installing GSAS-II for Mac with a single download" instructions for greater detail on these steps, but note the referenced file should be replaced with g2conda-1.0.1-MacOSX-x86_64.sh.

Download: Mac OS X: https://anl.box.com/v/g2conda101-mac64 (64-bit)

Execute the downloaded file:

    bash <path>/g2conda-1.0.1-MacOSX-x86_64.sh

Press enter to read through the copyright notice and type "yes" to accept it.
Enter a location for where the files should be installed, if the default is not OK.
- Enter a blank line for the proxy, unless your site requires one (not common)

After the installation is complete, the newly created GSASII.app file will be displayed in a finder window. Double-click on this to start this GSAS-II installation; drag it to the doc or create an alias. Note that an alias can be renamed, but renaming the GSASII.app file can cause problems.

Windows 64 build commands

rem ========== Configure
set condaHome=C:\jenkins\conda37-64
set sysType=win-64
rem ========== Anaconda stuff
call %condaHome%\Scripts\activate
REM call conda update conda -y
REM call conda create -n py37n119 python=3.7 numpy=1.19 matplotlib=3.3 wxpython=4 conda scipy pillow svn imageio anaconda-client constructor conda-build pyopengl h5py hdf5 pywin32 scons m2w64-gcc-fortran -y 
call %condaHome%\Scripts\activate py37n119
REM call conda install m2w64-gcc-fortran -y
REM call conda install -c msys2 m2w64-ntldd-git
REM call conda list
REM ====================================================================================
REM rmdir .\GSASII /s /q
REM mkdir .\GSASII
rmdir .\GSASII\AllBinaries /s /q
mkdir .\GSASII\AllBinaries
svn co https://subversion.xray.aps.anl.gov/pyGSAS/trunk/fsource .\GSASII\fsource
cd GSASII\fsource
where gfortran
where scons
call scons -c
call scons install=T 
REM LIBGCC=T LIBGFORTRAN=T removed
cd ..
rem #=# find the build dir name; add info to Build notes
for /f "usebackq" %%i in ( `dir .\AllBinaries /ad /b` ) do set newdir=%%i
echo %newdir%
echo on
REM ========== Create a note on how binaries were created =======================
echo Built with Jeeves from %JOB_NAME% from G2 version %SVN_REVISION% on %NODE_LABELS% >> AllBinaries\%newdir%\Build.notes.txt
REM systeminfo | findstr /B /C:"OS Name" /C:"OS Version" >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f python | findstr /B /C:# /V >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f numpy | findstr /B /C:# /V  >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f m2w64-gcc-fortran | findstr /B /C:# /V >> AllBinaries\%newdir%\Build.notes.txt
REM ============ End of note on how binaries were created =======================
svn co -q https://subversion.xray.aps.anl.gov/pyGSAS/Binaries .\Binaries
rmdir .\Binaries\%newdir% /s
mkdir .\Binaries\%newdir%
copy /y .\AllBinaries\%newdir%\* .\Binaries\%newdir%
REM
rem== Note: see https://anaconda.org/msys2/m2w64-ntldd-git for a utility that can find dependencies.
rem conda install -c msys2 m2w64-ntldd-git
ntldd .\Binaries\%newdir%\*.pyd
rem== (Diffax: files libquadmath-0.dll, libwinpthread-1.dll)
REM dir C:\Jenkins\conda37-64\envs\py37n119\Library\mingw-w64\bin
REM dir C:\Jenkins\conda37-64\envs\py37n119\Library\mingw-w64\lib
REM dir %condaHome%\envs\py37n119\Library\mingw-w64\bin\lib*.dll
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgfortran*.dll .\Binaries\%newdir%
ntldd .\Binaries\%newdir%\*.dll
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgcc*.dll .\Binaries\%newdir%
REM copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgmp*.dll .\Binaries\%newdir%
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libquadmath*.dll .\Binaries\%newdir%
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libwinpthread*.dll .\Binaries\%newdir%
ntldd .\Binaries\%newdir%\*.dll
svn add --no-ignore .\Binaries\%newdir%
svn add --no-ignore .\Binaries\%newdir%\*
svn st .\Binaries\%newdir%
svn ci -m"Build with %JOB_NAME% on %NODE_LABELS%" --username <omitted> --password %JENKINS_PASS%  --non-interactive .\Binaries
rem =========== Build the conda package and upload
cd ..
cd GSAS-II\install
call conda build purge
python setversion.py
echo off
call conda build g2complete --output-folder C:\Jenkins\g2builds
rem =========== Build the self-installer & copy to web
del *.exe
call constructor --clean
call constructor g2full
for /f "usebackq" %%i in ( `dir gsas2full*.exe /b` ) do copy %%i gsas2full-Latest-Windows-x86_64.exe
for /f "usebackq" %%i in ( `dir gsas2full*.exe /b` ) do scp %%i <acct@node.aps.anl.gov>:/home/joule/SVN/subversion/pyGSAS/downloads

(PKG_full_Win64 2021-01-16 20:17:56)

Windows 32 build commands

rem ========== Configure
REM scp <acct@node.aps.anl.gov>:Miniconda3-latest-Windows-x86.exe .
REM mkdir C:\Jenkins\g2builds
dir C:\Jenkins\g2builds
set condaHome=C:\jenkins\conda37-32
set sysType=win-32
rem ========== Anaconda stuff
call %condaHome%\Scripts\activate
REM call conda update conda -y
REM call conda create -n py37n119 python=3.7 numpy=1.19 matplotlib=3.3 wxpython=4 conda scipy pillow svn imageio anaconda-client constructor conda-build pyopengl h5py hdf5 pywin32 scons m2w64-gcc-fortran libpython -y 
call %condaHome%\Scripts\activate py37n119
call conda install -c msys2 m2w64-ntldd-git
REM call conda list
REM ====================================================================================
REM rmdir .\GSASII /s /q
REM mkdir .\GSASII
rmdir .\GSASII\AllBinaries /s /q
mkdir .\GSASII\AllBinaries
svn co https://subversion.xray.aps.anl.gov/pyGSAS/trunk/fsource .\GSASII\fsource
cd GSASII\fsource
where gfortran
REM====>>REM call scons -c
call scons install=T 
REM removed LIBGFORTRAN=T LIBGCC=T
cd ..
rem #=# find the build dir name; add info to Build notes
for /f "usebackq" %%i in ( `dir .\AllBinaries /ad /b` ) do set newdir=%%i
echo %newdir%
echo on
REM ========== Create a note on how binaries were created =======================
echo Built with Jeeves from %JOB_NAME% from G2 version %SVN_REVISION% on %NODE_LABELS% >> AllBinaries\%newdir%\Build.notes.txt
REM systeminfo | findstr /B /C:"OS Name" /C:"OS Version" >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f python | findstr /B /C:# /V >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f numpy | findstr /B /C:# /V  >> AllBinaries\%newdir%\Build.notes.txt
call conda list -f m2w64-gcc-fortran | findstr /B /C:# /V >> AllBinaries\%newdir%\Build.notes.txt
REM ============ End of note on how binaries were created =======================
svn co -q https://subversion.xray.aps.anl.gov/pyGSAS/Binaries .\Binaries
rmdir .\Binaries\%newdir% /s
mkdir .\Binaries\%newdir%
copy /y .\AllBinaries\%newdir%\* .\Binaries\%newdir%
REM
rem== Note: see https://anaconda.org/msys2/m2w64-ntldd-git for a utility that can find dependencies.
rem conda install -c msys2 m2w64-ntldd-git
REM dir C:\Jenkins\conda37-32\envs\py37n119\Library\mingw-w64\bin
REM dir %condaHome%\Library\mingw-w64\bin
ntldd .\Binaries\%newdir%\*.pyd
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgfortran*.dll .\Binaries\%newdir%
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgcc*.dll .\Binaries\%newdir%
ntldd .\Binaries\%newdir%\*.dll
REM copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libgmp*.dll .\Binaries\%newdir%
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libquadmath*.dll .\Binaries\%newdir%
copy /y %condaHome%\envs\py37n119\Library\mingw-w64\bin\libwinpthread*.dll .\Binaries\%newdir%
ntldd .\Binaries\%newdir%\*.dll
svn add --no-ignore .\Binaries\%newdir%
svn add --no-ignore .\Binaries\%newdir%\*
svn st .\Binaries\%newdir%
svn ci -m"Build with %JOB_NAME% on %NODE_LABELS%" --username <omitted> --password %JENKINS_PASS%  --non-interactive .\Binaries
rem =========== Build the conda package and upload
cd ..
cd GSAS-II\install
call conda build purge
python setversion.py
echo off
call conda build g2complete --output-folder C:\Jenkins\g2builds
rem =========== Build the self-installer & copy to web
del *.exe
call constructor --clean
call constructor g2full
for /f "usebackq" %%i in ( `dir gsas2full*.exe /b` ) do copy %%i gsas2full-Latest-Windows-x86.exe
for /f "usebackq" %%i in ( `dir gsas2full*.exe /b` ) do scp %%i <acct@node.aps.anl.gov>:/home/joule/SVN/subversion/pyGSAS/downloads
exit

(PKG_full_Win32 2021-01-17 09:38:41)

MacOS 64 build commands

condaHome=$WORKSPACE/conda37
sysType=osx-64
#========== Anaconda stuff
#curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh > ./Miniconda3-latest.sh 
#bash ./Miniconda3-latest.sh -b -p $WORKSPACE/conda37
#rm ./Miniconda3-*.sh 
set +x
echo 'source $condaHome/bin/activate'
source $condaHome/bin/activate
# wx, pyopengl, scons not available with Py39 in defaults
#echo conda create -n py39n1.19 python=3.9 numpy=1.19 matplotlib=3.3 conda scipy pillow svn h5py imageio gfortran_osx-64 -y
#     conda create -n py39n1.19 python=3.9 numpy=1.19 matplotlib=3.3 conda scipy pillow svn h5py imageio gfortran_osx-64 -y
# cant live without scons
#echo conda install scons -n py39n1.19 -c defaults -c conda-forge
#     conda install scons -n py39n1.19 -c defaults -c conda-forge
echo 'conda activate py39n1.19'
conda activate py39n1.19

#
#echo conda update -n base -c defaults conda -y
#     conda update -n base -c defaults conda -y
#
#echo conda env remove -n py37n1.18 
#     conda env remove -n py37n1.18 
#
#echo conda create -n py37n1.18 python=3.7 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio python.app=2 anaconda-client constructor conda-build scons gfortran_osx-64 -y
#     conda create -n py37n1.18 python=3.7 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio python.app=2 anaconda-client constructor conda-build scons gfortran_osx-64 -y
#echo 'conda activate py37n1.18'
#conda activate py37n1.18
#conda info --envs
#==================================== Build of .so files ==================================
#ls -l  $CONDA_PREFIX/bin/*gfortran
# gfortran seems to be setup by conda
#ln -sf $CONDA_PREFIX/bin/x86_64-conda_cos6-linux-gnu-gfortran $CONDA_PREFIX/bin/gfortran
#
#
set -x
which python
#python -c 'import sys; print("py=",sys.version); import numpy; print("np=",numpy.__version__)'
#python -c 'import wx; print("wx=",wx.__version__); import matplotlib; print("mpl=",matplotlib.__version__)'
#=======================
# get source code
#=======================
rm -rf $WORKSPACE/GSASII
mkdir -p $WORKSPACE/GSASII
svn co https://subversion.xray.aps.anl.gov/pyGSAS/trunk/fsource $WORKSPACE/GSASII/fsource
########################
# start compile process
########################
gfortran -v
rm -rf GSASII/AllBinaries
cd $WORKSPACE/GSASII/fsource
gfortran -v
scons -c # cleanup old
set +x
#scons install=T FORTpath=$CONDA_PREFIX/bin
scons install=T LIBGFORTRAN=T  # seems OK on OSX
# document build
#=# find the build dir name; add info to Build notes
set -x
newdir=`cd ../AllBinaries/; ls -d mac*`
echo $newdir
echo "Built with Jeeves from $JOB_NAME from G2 version $SVN_REVISION on $NODE_LABELS" >> ../AllBinaries/$newdir/Build.notes.txt
sw_vers >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f python | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f numpy | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f gfortran_linux-64 | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
#================
# Upload binaries
#================
svn co -q https://subversion.xray.aps.anl.gov/pyGSAS/Binaries $WORKSPACE/Binaries
if [ ! -d $WORKSPACE/Binaries/$newdir ]
then
    mkdir $WORKSPACE/Binaries/$newdir
    svn add $WORKSPACE/Binaries/$newdir
fi
python $WORKSPACE/Binaries/macRelink.py ../AllBinaries/$newdir
cp ../AllBinaries/$newdir/* $WORKSPACE/Binaries/$newdir
cp $CONDA_PREFIX/lib/libgcc_s.1.dylib $WORKSPACE/Binaries/$newdir/
cp $CONDA_PREFIX/lib/libquadmath.0.dylib $WORKSPACE/Binaries/$newdir/
svn add --force $WORKSPACE/Binaries/$newdir/*
svn st $WORKSPACE/Binaries
svn ci -m"Build with $JOB_NAME on $NODE_LABELS" --username <omitted> --password $JENKINS_PASS  --non-interactive $WORKSPACE/Binaries
#
exit

#=========================== Build of g2complete package ==================================
#
#=============================
# Use commands below to build
#=============================
cd $WORKSPACE/GSAS-II/install
python setversion.py
mkdir -p ~/builds
rm -rf ~/builds/*
set +x
echo conda build purge
     conda build purge
echo conda build g2complete --numpy 1.18 --python 3.7 --output-folder ~/builds
     conda build g2complete --numpy 1.18 --python 3.7 --output-folder ~/builds
#
#=========================== Build/upload of g2full installer =============================
#
# Build the self-installer
rm -f *.sh
constructor g2full
set -x
ls -l *.sh
# copy to "Latest" and upload
cp gsas2full-*-MacOSX-x86_64.sh gsas2full-Latest-MacOSX-x86_64.sh
# copy to https://subversion.xray.aps.anl.gov/admin_pyGSAS/downloads/gsas2full-*-MacOSX-x86_64.sh
scp gsas2full-*.sh <acct@node.aps.anl.gov>:/home/joule/SVN/subversion/pyGSAS/downloads/
echo files found at https://subversion.xray.aps.anl.gov/admin_pyGSAS/downloads/gsas2full-*-MacOSX-x86_64.sh
ssh <acct@node.aps.anl.gov> ls -lt /home/joule/SVN/subversion/pyGSAS/downloads/gsas2full-*-MacOSX-*.sh

(PKG_full_MacX86 2021-01-08 15:39:43)

Linux 64 build commands

condaHome=/Jenkins/jenkins/conda37
sysType=linux-64
#========== Anaconda stuff
#curl https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh > ./Miniconda3-latest-Linux-x86_64.sh 
#bash ./Miniconda3-latest-Linux-x86_64.sh -b -p /Jenkins/jenkins/conda37
#rm ./Miniconda3-*.sh 
set +x
echo 'source $condaHome/bin/activate'
source $condaHome/bin/activate
# wx not available with Py38
#echo conda create -n py38n1.18 python=3.8 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio anaconda-client constructor conda-build scons gfortran_linux-64 -y
#     conda create -n py38n1.18 python=3.8 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio anaconda-client constructor conda-build scons gfortran_linux-64 -y
#conda info --envs
#echo 'conda activate py38n1.18'
#conda activate py38n1.18
#
#echo conda update -n base -c defaults conda -y
#     conda update -n base -c defaults conda -y
#
#echo conda env remove -n py37n1.18 
#     conda env remove -n py37n1.18 
#
#echo conda create -n py37n1.18 python=3.7 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio anaconda-client constructor conda-build scons gfortran_linux-64 -y
#     conda create -n py37n1.18 python=3.7 numpy=1.18 matplotlib=3.3 wxpython=4.0 conda scipy pillow pyopengl svn h5py imageio anaconda-client constructor conda-build scons gfortran_linux-64 -y
#conda info --envs
echo 'conda activate py37n1.18'
conda activate py37n1.18
which gfortran
ls -l  $CONDA_PREFIX/bin/x86_64-conda_cos6-linux-gnu-gfortran
ln -sf $CONDA_PREFIX/bin/x86_64-conda_cos6-linux-gnu-gfortran $CONDA_PREFIX/bin/gfortran
which gfortran
#
#
set -x
which python
python -c 'import sys; print("py=",sys.version); import numpy; print("np=",numpy.__version__)'
python -c 'import wx; print("wx=",wx.__version__); import matplotlib; print("mpl=",matplotlib.__version__)'
#=======================
# get source code
#=======================
#DEBUG#
rm -rf $WORKSPACE/GSASII
mkdir -p $WORKSPACE/GSASII
svn co https://subversion.xray.aps.anl.gov/pyGSAS/trunk/fsource $WORKSPACE/GSASII/fsource
########################
# start compile process
########################
gfortran -v
rm -rf GSASII/AllBinaries
cd $WORKSPACE/GSASII/fsource
which gfortran
gfortran -v
#DEBUG#
scons -c # cleanup old
set +x
#DEBUG#
scons install=T FORTpath=$CONDA_PREFIX/bin
#scons install=T LIBGFORTRAN=T  # does not work with Conda gfortran
# document build
#=# find the build dir name; add info to Build notes
set -x
newdir=`cd ../AllBinaries/; ls -d linux*`
echo $newdir
echo "Built with Jeeves from $JOB_NAME from G2 version $SVN_REVISION on $NODE_LABELS" >> ../AllBinaries/$newdir/Build.notes.txt
cat /etc/system-release >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f python | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f numpy | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
conda list -f gfortran_linux-64 | tail -1 >> ../AllBinaries/$newdir/Build.notes.txt
#================
# Upload binaries
#================
svn co -q https://subversion.xray.aps.anl.gov/pyGSAS/Binaries $WORKSPACE/Binaries
if [ ! -d $WORKSPACE/Binaries/$newdir ]
then
    mkdir $WORKSPACE/Binaries/$newdir
    svn add $WORKSPACE/Binaries/$newdir
    cp ../AllBinaries/$newdir/* $WORKSPACE/Binaries/$newdir
    cp $condaHome/lib/libgfortran.so.4.0.0 $WORKSPACE/Binaries/$newdir/
else
    cp ../AllBinaries/$newdir/* $WORKSPACE/Binaries/$newdir
    cp $CONDA_PREFIX/lib/libgfortran.so.4.0.0 $WORKSPACE/Binaries/$newdir/
fi
svn add --force $WORKSPACE/Binaries/$newdir/*
svn st $WORKSPACE/Binaries
#DEBUG#
svn ci -m"Build with $JOB_NAME on $NODE_LABELS" --username <omitted> --password $JENKINS_PASS  --non-interactive $WORKSPACE/Binaries

#=============================
# Use commands below to build
#=============================
cd $WORKSPACE/GSAS-II/install
python setversion.py
mkdir -p ~/builds
set +x
echo conda build purge
     conda build purge
echo conda build g2complete --numpy 1.18 --python 3.7 --output-folder ~/builds
     conda build g2complete --numpy 1.18 --python 3.7 --output-folder ~/builds
#=========== Build the self-installer & copy (someday upload?)
rm -f *.sh
constructor g2full
set -x
ls -l *.sh
#=============================
cp gsas2full-*-Linux-x86_64.sh gsas2full-Latest-Linux-x86_64.sh
scp gsas2full-*.sh <acct@node.aps.anl.gov>:/home/joule/SVN/subversion/pyGSAS/downloads/
ssh <acct@node.aps.anl.gov> ls -lt /home/joule/SVN/subversion/pyGSAS/downloads/gsas2full-*-Linux-*.sh

(PKG_full_Linux64 2021-01-08 15:44:21)

Download in other formats:

Plain Text