[SOLVED] Is setting the linux memory to unlimit will have an adverse effect?

Issue

I am running MPI job in linux server. I got error:

--------------------------------------------------------------------------
The OpenFabrics (openib) BTL failed to initialize while trying to
allocate some locked memory.  This typically can indicate that the
memlock limits are set too low.  For most HPC installations, the
memlock limits should be set to "unlimited".  The failure occured
here:

Local host:    yw0431
OMPI source:   ../../../../../ompi/mca/btl/openib/btl_openib_component.c:1216
Function:      ompi_free_list_init_ex_new()
Device:        mlx4_0
Memlock limit: 65536

You may need to consult with your system administrator to get this
problem fixed.  This FAQ entry on the Open MPI web site may also be
helpful:

 http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages 
 --------------------------------------------------------------------------
 --------------------------------------------------------------------------
WARNING: There was an error initializing an OpenFabrics device.

Local host:   yw0431
Local device: mlx4_0
--------------------------------------------------------------------------
[yw0431:20193] 11 more processes have sent help message help-mpi-btl-openib.txt / init-fail-no-mem
[yw0431:20193] Set MCA parameter "orte_base_help_aggregate" to 0 to see all help / error messages
[yw0431:20193] 11 more processes have sent help message help-mpi-btl-openib.txt / error in device init
 forrtl: error (78): process killed (SIGTERM)

it means that my linux server have locked memory with 65M, but my job needed more memory. I think 2G should be emough.

I have found a solution about ulimiting the memory:

 ulimit -l unlimited

But i am worried that i will cause system crash or some bad things happen.

so can i set "ulimit -l umlimited"?

Solution

When you set ulimit as unlimited and your process starting using memory exhaustively then OOM killer will kill ur job for system stability,I would set the ulimit as 80 to 90% of RAM of instead of unlimited.

Answered By - Madhan S

Answer Checked By - Senaida (WPSolving Volunteer)