FAQ - Frequently Asked Questions

Latest version of this FAQ

The most recent version of this FAQ can always be found here.

Supported Compilers and Platforms

The following compilers have been tested in different STXXL configurations. Other compilers might work, too, but we don't have the resources (systems, compilers or time) to test them. Feedback is welcome.

Please note that from STXXL 1.4.0 on, only 64-bit systems are fully supported. Compilation on 32-bit seems to work, but we cannot support it anymore.

The compilers marked with '*' are the maintainers' favorite choices and are most thoroughly tested.

compiler	supported options
gcc 4.8.2	stxxl parallel (boost) (c++0x)
gcc 4.7.3 *	stxxl parallel (boost) (c++0x)
gcc 4.6.4	stxxl parallel (boost) (c++0x)
gcc 4.5.4	stxxl parallel (boost) (c++0x)
gcc 4.4.7	stxxl parallel (boost) (c++0x)
gcc 4.3.6	stxxl (boost)
gcc 4.1.2	stxxl (boost)
gcc 3.4.6	stxxl (boost)
gcc 3.3	unsupported
icpc 2013.5.192 *	stxxl (boost) (c++0x)
icpc 2011.13.367	stxxl (boost) (c++0x)
clang++ 3.1, 3.2, 3.3	stxxl (boost) (c++0x)
cygwin gcc 4.8.2	stxxl parallel (boost) (c++0x)
msvc 2013 12.0 *	stxxl (boost) (c++11)
msvc 2012 11.0	stxxl (boost) (c++0x)
msvc 2010 10.0	stxxl boost required

• The option "parallel" uses the __gnu_parallel extensions in some parts of STXXL. For all gcc versions >= 4.4 the __gnu_parallel extensions are ON by default. Support for MCSTL (predecessor of __gnu_parallel) was removed in STXXL 1.4.0.
• Boost is optional and not recommended on all systems, except MSVC 2010. It provides no advantages on other platforms.
  STXXL has been tested with Boost 1.40.0, 1.42.0 and 1.46.1. Other versions may work, too, but older versions will not get support.
• Support for C++0x and C++11 is integrated and automatically detected. No core parts of STXXL require C++11.
• All options are automatically detected by CMake.

How can I credit STXXL, and thus foster its development?

• For all users:
  • Sign up at Ohloh and add yourself as an STXXL user / rate STXXL: http://www.ohloh.net/p/stxxl
  • Rate STXXL at heise Software-Verzeichnis (German): http://www.heise.de/software/download/stxxl/76072
  • Rate STXXL at SourceForge: https://sourceforge.net/projects/stxxl/
• For scientific work: Cite the papers mentioned here: http://stxxl.sourceforge.net/
• For industrial users: Tell us the name of your company, so we can use it as a reference.

References to Elements in External Memory Data Structures

You should not pass or store references to elements in an external memory data structure. When the reference is used, the block that contains the element may be no longer in internal memory.
Use/pass an iterator (reference) instead.
For an stxxl::vector with n pages and LRU replacement strategy, it can be guaranteed that the last n references obtained using stxxl::vector::operator[] or dereferencing an iterator are valid.

However, if n is 1, even a single innocent-looking line like

std::cout << v[0] << " " << v[1000000] << std::endl;

can lead to inconsistent results.

Parameterizing STXXL Containers

STXXL container types like stxxl::vector can be parameterized only with a value type that is a POD (i. e. no virtual functions, no user-defined copy assignment/destructor, etc.) and does not contain references (including pointers) to internal memory. Usually, "complex" data types do not satisfy this requirements.

This is why stxxl::vector<std::vector<T> > and stxxl::vector<stxxl::vector<T> > are invalid. If appropriate, use std::vector<stxxl::vector<T> >, or emulate a two-dimensional array by doing index calculation.

Thread-Safety

The I/O and block management layers are thread-safe (since release 1.1.1). The user layer data structures are not thread-safe.
I.e. you may access different STXXL data structures from concurrent threads without problems, but you should not share a data structure between threads (without implementing proper locking yourself).
This is a design choice, having the data structures thread-safe would mean a significant performance loss.

Disk Allocation on Multiple Disks

I have configured several disks to use with STXXL. Why does STXXL fail complaining about the lack of space? According to my calclulations, the space on the disks should be sufficient.

This may happen if the disks have different size. With the default parameters STXXL containers use randomized block-to-disk allocation strategies that distribute data evenly between the disks but ignore the availability of free space on them.

STXXL in a Microsoft CLR Library

From STXXL user Christian, posted in the forum:

Precondition: I use STXXL in a Microsoft CLR Library (a special DLL). That means that managed code and native code (e.g. STXXL) have to co-exist in your library.

Symptom: Application crashes at process exit, when the DLL is unloaded.

Cause: STXXL's singleton classes use the atexit() function to destruct themselves at process exit. The exit handling will cause the process to crash at exit (still unclear if it's a bug or a feature of the MS runtime).

Solution:

1.) Compiled STXXL static library with STXXL_NON_DEFAULT_EXIT_HANDLER defined.

2.) For cleanup, stxxl::run_exit_handlers() has now to be called manually. To get this done automatically:

Defined a CLI singleton class "Controller":

public ref class Controller {
private: 
    static Controller^ instance = gcnew Controller;
    Controller();
};

Registered my own cleanup function in Controller's constructor which will manage to call stxxl::run_exit_handlers():

#pragma managed(push, off)
static int myexitfn()
{
    stxxl::run_exit_handlers();
    return 0;
}
#pragma managed(pop)

Controller::Controller()
{
    onexit(myexitfn);
}