JPL DARTS Lab Q&A - Recent questions and answers

Answered: How do I get back missing NFS mounts?

Tue, 23 Jul 2024 17:58:51 +0000

This can happen when the automounter dies. This can be fixed by simply restarting it with:

sudo service autofs stop
sudo service autofs start

Answered: How do I clear the php cache used by our apache web server?

Thu, 14 Mar 2024 22:09:07 +0000

You can purge the php cache on our web server (einstein at time of writing) by running with sudo:

systemctl reload php-fpm

Answered: How do I restart the GitLab elastic search?

Tue, 12 Mar 2024 18:14:53 +0000

From Boltzmann run

sudo systemctl restart elasticsearch

Answered: How do I restart the GitLab PlantUML server?

Tue, 12 Mar 2024 18:11:13 +0000

From Hamilton run

sudo /etc/gitlab/start_plantuml-server

Answered: How do I restart the pyam mysql server?

Wed, 31 Jan 2024 14:35:41 +0000

On einstein run

sudo systemctl restart mariadb

Answered: How can I convert a regular python script into a doctest script?

Tue, 21 Nov 2023 17:05:32 +0000

Given a doctest script.py, you can convert it into a regular script using

srun doctest-converter --to-doctest --input-file-name script.py --output-file-name doctest.py

This will create a doctest script called doctest.py.

Answered: My VPN is connected but I can't access any JPL websites or machines.

Wed, 15 Nov 2023 20:08:44 +0000

If you can connect to the VPN, but cannot connect to any JPL machines or websites, you may have a DNS issue. On Fedora, you may be able to resolve this by doing the following

1. Edit /etc/nsswitch.conf 


2. Comment out this line: hosts: files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] 

i.e. make  that line

# hosts: files myhostname mdns4_minimal [NOTFOUND=return] resolve [!UNAVAIL=return] 


3. Replace with this line: dns hosts: files myhostname mdns4_minimal resolve dns

Answered: How do I restart the svn server?

Fri, 27 Oct 2023 14:24:06 +0000

On einstein, run

svnserve -d -r /alt

Answered: How do we use glade to generate gui code?

Mon, 25 Sep 2023 18:17:55 +0000

glade is a tool for interactively generating GTK widget-based GUIs. It allows you to create windows and widgets, adjust their layout and properties, and assign callback methods to the various events supported by the widgets. It saves the GUI information in an XML object, which can be directly built and displayed using the GTK Python library. This process is very intuitive and clean, and is a great way to create GUIs.

Usage

Use glade-3 to create a simple GUI. I would recommend watching/following along to a YouTube tutorial to get the hang of how everything is laid out. Kevin O'Kane has a great playlist that helped me get going. In short:
1. Add a Window object and within it, put some Containers (Boxes, Grids, etc.). Then, you can select which type of element you want to add and add it to those containers. You can nest containers as much as you want.
2. Specify signal handler names under the Signal tab for GUI components you wish to be interactive.
3. Saving the GUI in glade will generate a .ui XML file which we will feed into the GTK Python library.
In Python, create a class with all the signal handlers. Make sure all the signal handlers are in the one class, and that the names match exactly. This class can also have other helper functions - i.e. it is not strictly limited to just the signal handlers.
In Python main, create the builder object and connect its signals to the class created in step 2. For example:

import gi
gi.require_version("Gtk", "3.0")
from gi.repository import Gtk
# build the GUI from the .ui file created using Glade
builder = Gtk.Builder()
builder.add_from_file("file.ui")

# connect the signals from the GUI elements to the corresponding methods in GUIController
gui_controller = GUIController()
builder.connect_signals(gui_controller)
# show the window
window = builder.get_object("window")
window.show_all()
Gtk.main()

As you make updates to the .ui file in glade, the GTK builder will automatically update the GUI when you rerun the program, making incremental development and testing very easy. You can also change, add, and remove GTK components in the GUI programmatically, meaning that you can make custom effects and behavior on the fly.

For a sophisticated example of this process in use in DARTS, take a look at the movie maker source code, located under FacadeLogScene/movieMaker. guiController.py contains the GUIController class that implements the signal handlers for the GUI.

Answered: How do I add the JPL root certificate to my laptop?

Mon, 18 Sep 2023 23:17:43 +0000

Copy JPLICA.Issuing1.pem and JPLICA.Root.pem from the /home/dartsfn/pkgs/etc directory of a JPL machine to your laptop at /etc/pki/ca-trust/source/anchors.
Run

sudo update-ca-trust

Restart your web browser.

Are there examples of integrating DARTS with MONTE for simulating autonomous navigation?

Mon, 14 Aug 2023 16:02:17 +0000

I would like to see examples of DARTS+MONTE integration for simulating autonomous navigation to a body (e.g., a moon, a planet, or a small-body asteroid).

Answered: How do I checkout a module on the main branch after saving it from a regtest sandbox?

Fri, 07 Jul 2023 20:52:42 +0000

Use the -w option. pyam save -w X will save the module and check it out as a work module as well after that.

Answered: How do I check how much space I (or a Team) have left in office365 (onedrive/sharepoint)?

Fri, 07 Jul 2023 18:42:07 +0000

The following instructions are valid as of 7/7/2023 and explain how to view storage usage and storage capacity for an Office365 account (i.e. OneDrive or Team) via the browser.

1. Log into OneDrive (https://jpl365prod-my.sharepoint.com/). You can also access this from Outlook, or other office365 access application by selecting "OneDrive" from the application menu.

2. Select "Return to classic OneDrive" in the lower left-hand corner. I have not found a way to view the usage metrics in the new OneDrive interface.

3. Select the settings icon on the top right of the screen, and select "Site settings" from the menu.

4. On the Site Settings page, find and select "Storage Metrics".

5. You should be able to view your individual usage metrics, including the total storage used and your storage capacity in the top right corner.

To view the storage metrics for a specific Team, the process is similar. However, once you've landed on the classic OneDrive page, you'll need to select the Team in question before selecting "settings -> site settings" as before. If you see no option to click "settings -> site settings from the Team page, you have likely been returned to the modern interface. Check the lower left corner for a "return to classic OneDrive" or "return to classis SharePoint", select this, and try again. Note: you will only be able to view a Team's usage metrics if you are a Team admin.

Answered: How do we handle loops in model data flow when using the assembly builder?

Sat, 10 Jun 2023 15:25:58 +0000

The error occurs because sim.createAssemblies() by default locks the simulation object at the end, which kicks in the model sorting process which will error out when there are loops. The solution is to pass in the lock=False optional argument to disable the locking, then externally add the model breaks, and then to manually lock the simulation object. The pseudo code will look something like:

sim.createAssemblies(config, lock=False)
# can do this multiple times for different model pairs
<look up model pair, and make mdl1.addSortingBreak(mdl2) call.>
sim.lockObject()

Answered: How can I create a complie_commands.json file for my DARTS sandbox?

Fri, 09 Jun 2023 17:31:23 +0000

There is a make target you can run from the top level

make compile-commands

This make target will create a compile_command.json file for you. It uses compiledb, which does not require recompliing everything like bear does. In addition, this make target will create the compile commands for both work and link modules.

Answered: How do I connect to the CAE WENG servers?

Mon, 08 May 2023 15:07:49 +0000

If you are on a Windows or Mac, instructions on how to use WENG can be found here.

If you are on a Linux machine, you cannot use Microsoft Remote Desktop. Hence, you will need to use a different RDP application. One easy, free one to use is freerdp. This can be installed on Fedora using:

sudo dnf install freerdp

Once this is installed, you connect to the WENG machines using the xfreerdp command. For example, to connect to the GPU machines use

xfreerdp /u:<YOUR_JPL_USERNAME> /v:cae-weng-gpu.jpl.nasa.gov

You will be prompted for your password (use your LDAP password), and then you should log in. The first login will take a while since it sets things up for you the first time. Subsequent logins should be quicker.

If you want things in full-screen mode (the above does Windowed mode), you can use

xfreerdp /f /u:<YOUR_JPL_USERNAME> /v:cae-weng-gpu.jpl.nasa.gov

If you start in full-screen mode, then you can use Ctrl+Alt+Enter to toggle full-screen mode and windowed mode.

Answered: How do I import Python modules that use HDF5 when the DARTS libraries use a different HDF5 version?

Wed, 05 Apr 2023 22:53:45 +0000

You can modify the dlopen flags to import Python modules in a context where previously loaded symbols will not conflict. For example, suppose we want to import tables in this way. We can do

from contextlib import contextmanager
import os

@contextmanager
def use_local_symbols():
    dl_flags = sys.getdlopenflags()
    try:
        sys.setdlopenflags(os.RTLD_NOW | os.RTLD_DEEPBIND)
        yield 0
    finally:
        sys.setdlopenflags(dl_flags)

with use_local_symbols() as _:
    import tables

This temporarily modifies the dlopen flags (via the Python context) in the following way:

RTLD_NOW resolves all symbols before dlopen returns
RTLD_DEEPBIND places the symbols for this object before global, so it will use its own symbols rather than the global ones.

Answered: How do I implemented doc tests in a Dclick file-exec script?

Thu, 16 Feb 2023 15:17:11 +0000

Suppose we want to run the following doc test

"""
x = 2+2

assert(x==4)
"""

The recipe to run this in a Dclick --file-exec script is:

doc_test = """
x = 2+2

assert(x==4)
"""

from tempfile import NamedTemporaryFile
import doctest, sys, os

tf = NamedTemporaryFile(suffix=".txt", dir=".")
with open(tf.name, "w") as f:
    f.write(doc_test)

(nfails, _) = doctest.testfile(
    tf.name,
    module_relative=False,
    verbose=os.getenv("DTEST_RUNNING") != "running",
    optionflags=doctest.NORMALIZE_WHITESPACE | doctest.ELLIPSIS,
    globs={"sim": sim},
)
sys.exit(int(nfails > 0))

This takes the doc string, stores it in a variable, writes it to a temporary file, and then runs doctest on that temporary file.

Answered: Is StateAttitudeRELFPAlignParam actually doing what it claims?

Tue, 14 Feb 2023 18:50:02 +0000

See APPENDIX H of the https://dartslab.jpl.nasa.gov/technotes/Releases/COMPASS/2019-03-25-R10/2019-06-27-COMPASS-Base-Package-R10-Delivery-Notes-v2.pdf for updates made by Clara OFarrell.

Is there a method for capturing a high/low watermark value for a variable and the associated time stamp?

Wed, 08 Feb 2023 18:35:51 +0000

I'm looking for a way to capture a high/low variable value with the associated timestamp, and without triggering an event?

Answered: How do I make GitLab use the full width of my screen?

Tue, 31 Jan 2023 19:24:59 +0000

1. Got to your preferences.

2. Scroll down to behavior and change the "Layout width" to "fluid".

Answered: How can multiple users share a common base sandbox?

Wed, 25 Jan 2023 22:53:47 +0000

The terms base and user sandbox will be used in this answer. The base sandbox refers to the common, shared sandbox among many users. The user sandbox refers to the individual, personal sandbox used by a user. The base sandbox contains common modules and code that users do not need to modify. The user sandbox contains modules with code that the user can run directly or change.

Base sandbox

The developer releasing the base sandbox should add the following to the site config:

USER_COMPILE_FLAGS=-I${BASE_SBOX}/include
USER_LINK_FLAGS=-L${BASE_SBOX}/lib/${YAM_TARGET}
CC_COMPILE_FLAGS              += $(USER_COMPILE_FLAGS)
CPLUSPLUS_COMPILE_FLAGS       += $(USER_COMPILE_FLAGS)
CC_STATIC_LINK_FLAGS          += $(USER_LINK_FLAGS)   
CPLUSPLUS_STATIC_LINK_FLAGS   += $(USER_LINK_FLAGS)
CC_SHARED_LINK_FLAGS          += $(USER_LINK_FLAGS)
CPLUSPLUS_SHARED_LINK_FLAGS   += $(USER_LINK_FLAGS)

This is the only thing that needs to change in the base sandbox.

User sandbox

The setup for the user sandbox is as follows:

1. Obtain a copy of the user sandbox.

2. Create an alias that points to the base sandbox's dlabrun_shared file. For example:

drun=<BASE_SBOX>/../dlabrun_shared

3. Run

drun make all

at the top level of the user sandbox.

Answered: How do I add new git servers to my gpg credentials?

Tue, 24 Jan 2023 23:31:09 +0000

Once you have gpg credentials setup, you can add new git servers to it by doing the following:

1. Decrypt the file so you can edit it.

gpg --decrypt --recipient <your_email> ~/.git-credentials.gpg > git-credentials

This will create a plain text file called git-credentials with all of your current git credential info.

2. Add new information using the same structure as before. For example, to add github.jpl.nasa.gov to your list of know credentials, append the following to the file.

machine github.jpl.nasa.gov

login YOUR_USERNAME

password YOUR_PASSWORD

protocol https

3. Re-encrypt your git credentials using

gpg --encrypt --recipient <YOUR_EMAIL> git-credentials && mv git-credentials.gpg ~/.git-credentials.gpg

4. Delete the unencrypted file.

rm git-credentials

Answered: How do I encrypt my git credentials with gpg?

Sat, 14 Jan 2023 18:00:46 +0000

1. Create unencrypted credentials file called git-credentials. with the following structure

machine dartsgitlab-internal.jpl.nasa.gov

login YOUR_USERNAME

password YOUR_PASSWORD

protocol https

2. Encrypt this file by doing

gpg --encrypt --recipient <YOUR_EMAIL> git-credentials && mv git-credentials.gpg ~/.git-credentials.gpg

If you do not have a GPG key, you will need to generate one with

gpg --generate-key

If you are using this for a cron job, you should not set a password. That way, the cron job can decrypt the file without an extra password.

3. Check that this worked correctly by doing

gpg --decrypt --recipient <YOUR_EMAIL> ~/.git-credentials.gpg | diff -s git-credentials -

4. Delete the original, unencrypted file.

rm git-credentials

5. Get the git-credential-netrc.perl script put it somewhere on your PATH. This script is located here. You can use the following command to save it to some place on your path and make executable (saved to /home/dlab/pggs/bin on our system):

curl -o ./git-credential-netrc.perl https://raw.githubusercontent.com/git/git/master/contrib/credential/netrc/git-credential-netrc.perl

6. To tell git to use these gpg credentials, run

git config --global credential.helper 'netrc -f ~/.git-credentials.gpg'

7. To check that git has switched to gpg run

git config --global --list

It should show the 'credential.helper=netrc -f ~/.git-credentials.gpg' line in the output

Answered: How does one get plantUML working on the gitlab server?

Fri, 13 Jan 2023 06:44:09 +0000

On hamilton, run /etc/gitlab/stop_plantuml-server to stop the server, and /etc/gitlab/start_plantuml-server to start the plantUML server.

Answered: How do I add a self-signed certificate to git?

Thu, 12 Jan 2023 15:44:29 +0000

DARTS lab machines

If you're on a dlab, machine then all you need to do is run

git config --global http.sslCAInfo /home/dartsfn/pkgs/etc/cert.perm

This will add the common SSL certificates that we use. To add a new certificate to the common set, you need to modify the script that auto-generates certificates each day via the ssl_certs job. Servers can be added to the list in /home/dartsfn/pkgs/etc/generate_cert.py

Local machine and custom SSL servers

If you need to run things from your local machine, you can copy the cert.perm file specified above to a location and then set git to use it. The instructions below can be used to add extra servers to a cert.perm file.

There are three steps needed to do this:

Get the certificate.
Save it to your computer in a file.
Set git to trust certificates in this file.

Get the certificate and save it to your computer

You can obtain the certificate from your web browser.

In Chrome, this can be done by doing the following:

Got to the main https://dartsgitlab-internal.jpl.nasa.gov site
Click the Secure button (a padlock) in an address bar. If using JPL servers, this may be a warning sign that says "Not secure".
Click the Certificate(Valid). Again, if using JPL servers this may say "Certificate is not valid."
Go to the Details tab.
Click the "Export ..." button, and save it to a file.

In Firefox, this can be done by doing the following:

Click the Secure button (a padlock) in an address bar.
Click the right arrow next to "Connection secure." If using a JPL server, this may say "Connection not secure."
Click the "More information" button.
Click "View certificate."
Click the "Download" button to save it to a file.

You can also retrieve it using Python by running

import ssl
ssl.get_server_certificate((server, 443))

where server is the name of the server. For example, for the JPL GitHub, this would be:

ssl.get_server_certificate(('github.jpl.nasa.gov', 443))

Set git to trust certificates in this file

Run the following command:

git config --global http.sslCAInfo <path_to_cert_file>

In the previous step, suppose you saved this file to ~/.git-certs/cert.perm, then the command would be:

git config --global http.sslCAInfo ~/.git-certs/cert.perm

Answered: How do I deliver a DSENDS package as a github repo?

Thu, 01 Dec 2022 00:34:21 +0000

For the DsendsSRLPkg delivery to nexus, see:

https://dartslab.jpl.nasa.gov/technotes/Talks/2022-03-24%20Moving%20DsendsSRLPkg%20to%20Git.pdf

Answered: How do I query part geometries and turn their visibility off?

Wed, 02 Nov 2022 02:11:51 +0000

Part geometries for a particular body are accessed from the body.

For example, given an instance of a body, bd, to view all part geometries associated with that body:

bd.partGeometrySpecs().keys()

To toggle off the visibility for a particular part geometry:

bd.getPartGeometry('visual').visibility(False)

Answered: How can I share a remote terminal session with other developers?

Fri, 28 Oct 2022 14:20:09 +0000

The tmate tool is very handy for sharing terminal sessions. The tool can be readily installed using apt, dnf or any other package manager.

Now, lets say user A wants to get help from user B to debug a problem on A's computer. In this case user A may find it useful to share a terminal with B so that B can use the keyboard to work directly on it. For this, A should run tmate in a local terminal, and cut and paste the keys it prints out in a slack message to B. B can can then simply run ssh <key> (normally the last of the 4 keys) on a local terminal. After this both A and B can type in their local terminals and see each other's output as for a shared terminal. The only thing missing will be mousing which would require a VNC session. However a shared terminal can be very useful in its own right.

How do I model a spacecraft bus with two solar arrays connected by a flexible hinge

Thu, 08 Sep 2022 15:25:26 +0000

I have a spacecraft that I model as a rectangular bus with two large solar panels. I want to allow the joint to be flexible and have been looking at using the SpringDamperMotor model.

I essentially want to have a flexible 'BALL' joint, meaning the joint can flex up/down and left/right, along with twist.

Is there an example of this? I think the SpringDamperMotor model is a single-axis motor (all the examples are 'PIN' joints and robot arms), is there an example of a more complicated flexible joint?

Answered: How does one profile C++ code called from a Python script?

Wed, 17 Aug 2022 17:33:50 +0000

Usage

The google profiling tools and yep Python library can be used to do this.

In the Python code do:

import yep
yep.start("my_profile.prof")
# code you want to run goes here
yep.stop()

The output file, my_profile.prof, can then be analyzed using the google pprof tool. Note, this is different than the regular pprof tool, and has a different name on the dlab machines: gpprof. To analyze the output file simply run (on the dlab machines)

gpprof my_profile.prof

This will open the my_profile.prof file in the google profiler. Then, simply run one of the commands, e.g., web, to view the output. See page for example output.

Information on how to interpret the graph can be found here.

Wall time

By default, yep uses cpu time. However, in some instances, e.g., mutlithreading, it can be more useful to measure wall time. This can be done by setting the following environment variable:

CPUPROFILE_REALTIME=1

e.g., in bash this can be done with

export CPUPROFILE_REALTIME=1

Installation

The yep Python library can be installed via pip, i.e., pip install yep
The google profiler can be installed through the package manager. For example, on Fedora it is dnf install golang-github-google-pprof. Note, this may cause a name conflict if pprof is already installed. In that case, I recommend renaming the current pprof to temp_pprof. Install the library, rename the google pprof to gpprof, and then move temp_pprof back to pprof. In the end, this will keep the original pprof with the original name, and install the google pprof as gpprof.

Answered: How do I set up VSCode to start working with a DARTS sandbox?

Mon, 04 Jul 2022 17:05:14 +0000

Useful extensions

Extensions can be installed to add features to VSCode. These are some of the extensions that are useful when working on DARTS code. To install an extension (or a set of extensions), open the "Extensions" panel on the left (Ctrl+Shift+X) and search for the name of the extension.

Python: this is a set of extensions that enables features like type hinting, linting, jupyter notebooks renderers... Extremely recommended for any developer of python code.
- Additionally, once this set is installed, I recommend going to the "Pylance" page in the "Extensions" marketplace, clicking on the "Manage" cog symbol below the description of Pylance, clicking on "Extension settings", and then finding "Python > Analysis > Type Checking Mode" and switching from "off" to "basic". This will tell VSCode to add squiggly error lines under python code that Pylance finds incorrect (such as adding a string and a float).
autoDocstring: allows autogenerating python docstrings for functions and classes. Since we use numpy-style docstrings, I recommend going into the "Extension settings" and changing "Auto Docstring: Docstring Format" to "numpy"
AsciiDoc: useful for writing documentation files in AsciiDoc format in a local file before uploading to GitLab. Allows live-preview of how the AsciiDoc would look and syntax highlighting.

Answered: How can edit and modify the contents of an HDF5 file?

Sat, 25 Jun 2022 02:29:47 +0000

Lets say the file is foo.h5. First thing to do is to view its contents using

HDFView foo.h5

Browse through the content to find the dataset you want to modify and note the path for it, eg. '_dbase/object_43File/ints/_type'. Now fire up python and at is prompt run

import h5py
file = h5py.File('HDF5Store', mode='r+')
h = file['_dbase/object_43File/ints/_type']  # look up the dataset
h['<data key>'][...]   # to examine the data (the ... are important)
h['<data key>'][...] = 5   # set the scalar data
file.close()     # save the data

Run HDFView again to verify that the data has changed.

Answered: How do I convert GitLab flavored markdown to a PDF?

Fri, 24 Jun 2022 21:40:11 +0000

There are many ways to convert markdown to a PDF. However, most of these do not work with GitLab flavored markdown. The best way I have found is to make use of Python's markdown and a few extensions to convert to HTML, and then use weasyprint to convert the HTML to a PDF. Specifically, we will be using the extra and katex extensions.

Example Python conversion script:

import markdown
from markdown_katex import KatexExtension
from weasyprint import HTML

markdown.markdownFromFile(input='test.md', output='test.html', extensions=[KatexExtension(), 'extra'])
HTML("test.html").write_pdf("test.pdf")

Answered: How do I set up my non-JPL account on the dartsgitlab.jpl.nasa.gov site?

Mon, 20 Jun 2022 12:41:08 +0000

Check that you can access https://einstein.jpl.nasa.gov. If not send your IP address (or a range) so they can be whitelisted. Once you can access the page:

1. Click Standard Tab at the login screen

2. Enter your specified username with no quotes.

3. Enter default password and choose new password and validate then will return to the login screen

4. Log in again with your user name and password chosen in step 3.

5. Will specify that two-factor authentication is required and will need to set that up

6. User will need to download 2FA app like "Google Authenticator" with steps shown.

Answered: How do I convert Markdown to Asciidoc?

Tue, 24 May 2022 22:04:59 +0000

Although pandoc can be used to convert Markdown to Asciidoc, I found that pandoc sometimes had issues resolving embedded lists. In my experience, Kramdoc works much better; especially for GitLab flavored Markdown.

Installation

To install Kramdoc run:

gem install kramdown-asciidoc

Usage

To use Kramdoc I recommend running the following:

kramdoc --format=GFM --output=<output_file_name.asciidoc> --wrap=preserve <input_file_name.md>

Extras

For more information on Kramdoc, see here.

Answered: How can I trigger deprecation warnings for a module variable?

Mon, 16 May 2022 20:29:51 +0000

This can be achieved through the deprecated_module_attributes method in Dutils.deprecated.

Imagine your module test_module.py used to have the module-level attributes OLD_VARIABLE and DELETED_VARIABLE. You wish to replace OLD_VARIABLE with NEW_VARIABLE, while DELETED_VARIABLE should not be used at all. However, you do not wish yet to completely remove OLD_VARIABLE and DELETED_VARIABLE as they might be in use by customers. Then, if your original file is:

# test_module.py

OLD_VARIABLE = 42
DELETED_VARIABLE = 101

then the new file should read:

# test_module.py
from Dutils.deprecated import deprecated_module_attributes

NEW_VARIABLE = 42

__getattr__ = deprecated_module_attributes(
    old_values = {
        "OLD_VARIABLE": NEW_VARIABLE,
        "DELETED_VARIABLE": 101
    },
    new_names = {
        "OLD_VARIABLE": "NEW_VARIABLE",
    }
)

If one imports either `OLD_VARIABLE` or `DELETED_VARIABLE`, then deprecation warnings will be displayed:

import test_module

print("--- Warnings on use:")
test_module.OLD_VARIABLE
test_module.DELETED_VARIABLE

print("--- Warnings on import:")
from test_module import OLD_VARIABLE
from test_module import DELETED_VARIABLE

which will show the following

--- Warnings on use:
~/script.py:4: DeprecationWarning: The variable OLD_VARIABLE is deprecated. Use the variable: NEW_VARIABLE.
  test_module.OLD_VARIABLE
~/script.py:5: DeprecationWarning: The variable DELETED_VARIABLE is deprecated.
  test_module.DELETED_VARIABLE
--- Warnings on import:
~/script.py:8: DeprecationWarning: The variable OLD_VARIABLE is deprecated. Use the variable: NEW_VARIABLE.
  from test_module import OLD_VARIABLE
~/script.py:9: DeprecationWarning: The variable DELETED_VARIABLE is deprecated.
  from test_module import DELETED_VARIABLE

How do I get plugins for my Unreal Engine project despite me running Linux?

Sat, 07 May 2022 01:06:41 +0000

Since we get plugins through the Epic Games store, we can't get plugins directly while using linux.

Instead, use your PC or Mac to get Epic Games Launcher and Unreal Engine. Buy various plugins and install them to either the engine or a project. Sometimes these plugins might add functionality or they may add art assets.

In the case that they add functionality, they will install themselves directly into the engine or your plugins folder.

If they install to the Engine, you need to navigate to the Engine plugins folder and find the folder with your plugins name and copy it into the linux Engine.
If they install to the project itself, in your project directory look for a folder called "plugins" and copy that folder into your linux unreal project.

In the case that they add art assets, they might either install itself into the engine or project, if so look above for how to deal with it.

If it imports art assets into your project (or makes its own), just copy that project into linux, open the project up using the linux unreal engine and migrate the files into the Unreal Engine project you're working in (i.e. UnrealDarts)

Warning! Migrating assets between projects may mess up your file hierarchy, so after you get your wanted assets be sure to move them (in the Unreal Engine) into their proper place. After moving it be sure to right-click the top-level Contents folder and "Fix up redirectors".

Also, every month Epic Games releases a bunch of free plugins. These plugins range from sound effects to art assets and programmed feature packs (like an inventory system). Might be something to check every month as these plugins can be very high quality.

Answered: How can I determine which events in the FSM have triggered and which have not?

Wed, 04 May 2022 22:50:15 +0000

To get the triggered/untriggered states for the current time:

fsm = sim.fsm()

# get all state names
[s.name() for s in fsm.allStates()]

# use fsm.event_table to get triggered states
triggered_states = [i.name() for i in fsm.event_table]

# get untriggered states
untriggered_states = []
for s in fsm.allStates():
   if s.name() not in triggered_states:
      untriggered_states.append(s.name())

If you want to get a history, you will need log the triggered/untriggered states in DataRecorder or Dwatch.

Answered: How do I use UnrealScene for development?

Tue, 03 May 2022 17:37:41 +0000

Working with multiple people (using source control)

Working with Unreal Engine projects with multiple people can be a headache if done improperly. Often times when people are working in projects they may be unknowingly working in the same maps or files.

Here's how you can best avoid merge conflicts inside of Unreal Engine and Blueprints.

C++ merge conflicts get handled the usual way.
Set the startup map of the project to be blank
- Blank map means no one can accidentally change things of the startup map.
Work in separate testing maps in a directory under _Maps/TestMaps
- i.e. MAP_Matthew, MAP_Carolina
If people are trying to add functionality to a blueprint, try to create a child class from a common blueprint first and work separately that way.
- When you want to merge your changes together you should be able to copy/paste your code into the blueprint.
- Risky, but an alternative: is to work in the same file, but before merging with upstream duplicate your conflicted files and add the postfix: _CONFLICT and then merge and override your conflicted files with their version and copy manually the blueprints from your _CONFLICT file.

Unreal Engine has source control functionality, but it's not useable for us because of how DARTS lab handles source control.

I need a frame that is always located at the CM but is fixed to the body

Thu, 21 Apr 2022 00:20:30 +0000

I need a frame that is always located at the center of mass of the main body but which is also fixed relative the body frame. I thought simObj.mbody().cmFrame() would do this, but it seems like it's fixed relative to the pcr frame. Will "CapsuleBase_obj103_cm_frame" or "cegraph_cm_frame" give me what I'm looking for? I can also create one, but it's not clear to me how to create one that has both of these properties (i.e. always at the cm and rotates with the body).

If the answer is CapsuleBase_obj103_cm_frame, then will the object number of this frame always be the same?

Answered: How do I improve exceptions thrown by SWIG directors?

Wed, 20 Apr 2022 18:03:56 +0000

The SWIG documentation recommends adding the following to improve SWIG director exception handling:

%feature("director:except") {
  if ($error != NULL) {
    throw Swig::DirectorMethodException();
  }
}

%exception {
  try { $action }
  catch (Swig::DirectorException &e) { SWIG_fail; }
}

WARNING: As is, this block will re-define exceptions for all functions after the exception block. This may lead to undesirable behavior. A better approach is to refine the scope of exception. For example, to apply it to a specific method of a class within a namespace do:

%feature("director:except") mynamespace::myclass::mymethod {
  if ($error != NULL) {
    throw Swig::DirectorMethodException();
  }
}

%exception mynamespace::myclass::mymethod {
  try { $action }
  catch (Swig::DirectorException &e) { SWIG_fail; }
}

This improves the error message by include the Python stack trace. For example, here is an error message before adding the improved exception handling:

File "/home/leake/Documents/eelsPkg/lib/PYTHON/DScene/DScene_Py.py", line 2239, in executeReverse
    return _DScene_Py.CallbackRegistry_executeReverse(self)                                                               
SystemError: SWIG director method error. Error detected when calling '_CallbackPyBase.execute'

Here is the same exception after adding the improved exception handling:

File "/home/leake/Documents/eelsPkg/lib/PYTHON/DScene/DScene_Py.py", line 2239, in executeReverse                           
    return _DScene_Py.CallbackRegistry_executeReverse(self)                                                                    
  File "/home/leake/Documents/eelsPkg/lib/PYTHON/DScene/DScene_Py.py", line 2401, in execute       
    self._f()                                                                                                                  
  File "test.py", line 30, in closeExtraDVars                                                                                  
    dv.thisown = True 
AttributeError: 'tuple' object has no attribute 'thisown'

The improved exception points to the exact line in Python that caused the error, whereas the original did not.

Answered: What are options for performing convex decomposition in DARTS, and how do they differ?

Tue, 05 Apr 2022 16:02:07 +0000

The following is a list of the available methods in DARTS:
1. MeshToBullet2. This script available with DBullet implements interfaces to the VHACD-4.0, VHACD-2.0, and HACD convex decomposition algorithms. For a more detailed discussion on these algorithms, see the DBullet wiki.

2. MeshToBullet. This binary file available with DBullet is a standalone script that can help user's generate convex hulls from mesh files; it uses Bullet's original "ConvexDecomposition" algorithm. In forms, example here, it's noted that this algorithm is a bit more intuitive in terms of the settings the user can tweak compared to HACD and VHACD-2.0; however, the author of this post feels the intuitiveness of the settings for the VHACD-4.0 algorithm is similar. Moreover, the algorithms behind MeshToBullet2 should yield better results than those in MeshToBullet for most cases. Another useful note here is DBullet itself now uses HACD as the default for convex decomposition, and PyBullet uses a version of VHACD.

The following is a list of methods that exist, but have not been utilized heavily yet:
1. The DMesh module can perform convex decompositions on DMeshes by using the "computeConvexDecomposition" method. This internally uses CGAL's Convex_decomposition_3 function.

Answered: How do I redirect C-level streams in Python?

Mon, 04 Apr 2022 21:04:30 +0000

There is a nice article here that gives a great explanation on how streams work at the system level, and why "redirect_stdout" does not work for C-level streams. Then, it discusses how to create a re-director that will redirect C-level streams. I am posting that code below for convenience:

from contextlib import contextmanager
import ctypes
import io
import os, sys
import tempfile

libc = ctypes.CDLL(None)
c_stdout = ctypes.c_void_p.in_dll(libc, 'stdout')

@contextmanager
def stdout_redirector(stream):
    # The original fd stdout points to. Usually 1 on POSIX systems.
    original_stdout_fd = sys.stdout.fileno()

    def _redirect_stdout(to_fd):
        """Redirect stdout to the given file descriptor."""
        # Flush the C-level buffer stdout
        libc.fflush(c_stdout)
        # Flush and close sys.stdout - also closes the file descriptor (fd)
        sys.stdout.close()
        # Make original_stdout_fd point to the same file as to_fd
        os.dup2(to_fd, original_stdout_fd)
        # Create a new sys.stdout that points to the redirected fd
        sys.stdout = io.TextIOWrapper(os.fdopen(original_stdout_fd, 'wb'))

    # Save a copy of the original stdout fd in saved_stdout_fd
    saved_stdout_fd = os.dup(original_stdout_fd)
    try:
        # Create a temporary file and redirect stdout to it
        tfile = tempfile.TemporaryFile(mode='w+b')
        _redirect_stdout(tfile.fileno())
        # Yield to caller, then redirect stdout back to the saved fd
        yield
        _redirect_stdout(saved_stdout_fd)
        # Copy contents of temporary file to the given stream
        tfile.flush()
        tfile.seek(0, io.SEEK_SET)
        stream.write(tfile.read())
    finally:
        tfile.close()
        os.close(saved_stdout_fd)

Answered: What are good practices for setting up a Kanban board on gitlab?

Fri, 01 Apr 2022 15:08:22 +0000

Kanban boards can be used for managing project issues, individual developer assigned issues etc. gitlab supports the creation of multiple Kanban boards that can be saved and revisited later. The key idea with a Kanban board is populate it with lists, with Open issues being on the left end, and Closed ones on the right end. The goal is to move issues progressively from the left to the right over time.

See the CADRE project Kanban board for an example of the ideas listed below for setting up Kanban boards.

Our gitlab projects most map into YaM modules. Project issues normally span multiple modules. So to manage a projects issues, create the Kanban board at a higher group level which contains all the module projects of interest. Good places would be the JPL Internal group level. Give the board a name that makes it easy to pick out, since we expect to have many boards over time.
Once you create the board, we want to narrow it down its issues to the ones relevant to the project. You can do this by going to the Edit board link on the upper right and using milestone, assignee etc filters to filter down the list of issues. For the CADRE board the issues are narrowed down to the ones tied to the CADRE v5 delivery gitlab milestone.
Now use labels to add lists to the board via the Create list link on the upper right. We want to keep the number of lists as small as possible to reduce clutter. Note that you can drag and drop issues across lists and this will automatically update the labels. Here is a suggested list in left to right order:
- stage::DEVELOP label: these are issues that are open, and ready to be worked on. That is, ones where we have sufficient description and understood approach.
- priority::HIGH label: these are the issues that are the highest priority among the list of develop ready issues.
- stage::INPROGRESS label: these are issues from the high priority ones that are actively being worked.
- stage::REVIEW label: these are issues on whom work has been completed, but are ready for review by the team before closing them out. This list can be dropped if there is only one member on the team.
- stage::ONHOLD label: these are issues that were in progress, but have been side tracked for now due to some circumstances with the expectation that they will be moved back to the stage::INPROGRESS stage in the near future.

Tips on periodically managing the boards:

Start with the open items on the left and see if any have evolved in their definition can can be moved to the develop list
Scan the develop list to see if any are priorities that should be moved to the high list on the right. Also, any items that are no longer a priority should be removed altogether, or moved to the right into the open items list.
Scan the high priority list and see which are the ones that you are ready to work on, and move them to the on progress list. This action should only be done by the developer actually doing the work. In general we expect there be very small number of in progress items for any one developer.
Scan your assigned in progress items, and see if they are completed and can be moved to the review list, or if you are stuck waiting on something and in which case they should be moved to the on hold list. Again these actions should only done by the developer actually doing the work.
At a team meeting, go over the issues in the review column and see if there is agreement that they are indeed done, and if so, close the item.

Answered: What are dlopenflags?

Tue, 29 Mar 2022 18:16:54 +0000

Answer

The dlopenflags are used to control the flags Python uses when making calls to dlopen. dlopen is used to open dynamic shared objects, e.g., shared libraries. These can be get/set using the sys module, e.g., sys.getdlopenflags().

In DARTS, these flags are set to os.RTLD_GLOBAL | os.RTLD_NOW, which means the shared object will be made available for symbol resolution of subsequently loaded shared objects' undefined symbols, and symbols in the shared object are resolved before dlopen finishes. For more information, see the man page on dlopen.

In DARTS, the Dshell++Scripts/DebugLog_Py.py module sets these values.

Corner cases

Sometimes, the global sharing of symbols can cause an issue. For example, if the Python noise package is loaded and used immediately after, intermittent seg faults can occur. The stack traces of these seg faults may point to missing files or missing symbols. In these cases, the opendlflags may need to temporarily be reset to os.RTLD_NOW for the import of such packages. See DMesh_Py.applySimplexNoise for an example.

Answered: How do I generate a terrain using simplex noise?

Fri, 18 Mar 2022 22:43:26 +0000

Breakdown

This question can be divided into two parts:

How do we generate simplex noise?
How do we apply this noise to a mesh and bring in into DARTS?

1. How do we generate simplex noise?

The Python noise package can be used to easily generate simplex noise. This article is also useful in helping understand the simplex noise parameters.

2. How do we apply this noise to a mesh and bring in into DARTS?

The DMesh::applySimplexNoise function can be used to apply simplex noise to a function. Moreover, DMesh meshes can easily be incorporated with the rest of the DARTS framework. The arguments for applySimplexNoise are the mesh that the noise should be applied to, and a layers argument that contains the simplex noise parameters. For more information on the layers argument, see the doc string for the applySimplexNoise function.

Alternative solution

Alternatively, one could write the noise function themselves and use the DMesh::Mesh::perturbMesh function, which can be used to perturb a mesh using a user-defined function. An example is given here using the Python noise package.

Example:

The following script shows an example:

import os
import noise
from DScene import DMesh_Py

# Constants

name = "NoisyTerrain"
x_len = 10
y_len = 10
numX = 40
numY = 40

layers = [
    dict(amp=4, seed=5, res=1, octaves=10, persistence=0.5, lacunarity=2.0),
]

# Calculated constants
size = max(x_len, y_len)

# Noise function
def add_simplex_noise(x, y, res, seed, amp=1, octaves=1, persistence=0.5, lacunarity=2.0):
    scale = size/res
    return amp*noise.snoise3(
        x/scale,
        y/scale,
        seed,
        octaves=octaves,
        persistence=persistence,
        lacunarity=lacunarity)

def generate_simplex(x,y,z):
    for layer in layers:
        amp = layer.get("amp", 1)
        res = layer.get("res", 1)
        octaves = layer.get("octaves",2)
        persistence = layer.get("persistence", 0.5)
        lacunarity = layer.get("lacunarity", 2.0)
        seed = layer.get("seed", 0)
        z += add_simplex_noise(x,y,res,seed,amp=amp, octaves=octaves, persistence=persistence,lacunarity=lacunarity)

    return x,y,z


# Create flat mesh
mesh = DMesh_Py.constructPlanarMesh(name, x_len, y_len, numX, numY)

# Perturb it using the noise function above
mesh.perturbMesh(generate_simplex)

# Visualize the mesh
mesh.show()

Answered: Given a tar ball on a telerbotics machine, How do you copy and run it on nexus?

Thu, 17 Mar 2022 19:36:33 +0000

To copy to nexus from a telerobotics machine:

scp /home/dlabdriveS/repo/docker/deliv/JplDartsLab-DsendsEnceladusPkg-R1-00-2022-03-16-mintps-nexus7.4_monte158.tar <your user name>@nexus:/home/<your user name>

For example, my user name is `cslim` so if I'm logged into hertz, the copy command would be:

scp /home/dlabdriveS/repo/docker/deliv/JplDartsLab-DsendsEnceladusPkg-R1-00-2022-03-16-mintps-nexus7.4_monte158.tar cslim@nexus:/home/cslim

On nexus, untar the tar ball

ssh -Y nexus

tar xvf JplDartsLab-DsendsEnceladusPkg-R1-00-2022-03-16-mintps-nexus7.4_monte158.tar

Use DsendsEnceladusPkg-R1-00-2022-03-16-monte158-nexus7.4_conda_mdnav_11.2_monte_158-PY3/dlabrun instead of srun to run scripts. For example:

DsendsEnceladusPkg-R1-00-2022-03-16-monte158-nexus7.4_conda_mdnav_11.2_monte_158-PY3/dlabrun python <your script.py>

Answered: How do I visualize a DAT file?

Tue, 08 Mar 2022 20:29:24 +0000

Gmsh

Gmsh is a free, open-source tool that can be used to view NASTRAN files.

To use it do:

Run "gmsh <filename>" from the terminal. Alternatively, just run "gmsh" and open the file via the GUI.
Turn on nodes and 1-D elements by going to Tools --> Options --> Mesh and check the appropriate boxes. Node labels (numbers) can also be enabled via this window.

Hypermesh

Hypermesh can also be used to visualize NASTRAN files. However, I found it less intuitive to use than gmsh. Also, see warning below.

WARNING: This is CAE software, specifically HyperWorks. If you use it you will likely get an email from the CAE General Community regarding the subscription fee, which at the time I am writing this answer is "$6.46 per work hour against the accounts on your timecard – about $221 per week. It is not an hourly use charge." This charge grants you access to all CAE tools.

To use it do:

Add the following to your .bashrc file:

alias hm="cd ~/2017 && /home/dlab/pkgs/src/altair/altair/scripts/hm"
export ALTAIR_LICENSE_PATH="6200@cae-lm-prod1.jpl.nasa.gov"

or if you are using tcshrc, use:

alias hm "cd ~/2017 && /home/dlab/pkgs/src/altair/altair/scripts/hm"
setenv ALTAIR_LICENSE_PATH "6200@cae-lm-prod1.jpl.nasa.gov"
This alias uses the ~/2017 as a temporary folder for all Hypermesh files. If you do not have a 2017 folder in your home directory, create one. If you want to use a different temporary folder name, create one.
From the terminal, type hm to start Hypermesh.
Import the DAT file to visualize.
- Choose Nastran from the User Profiles menu that opens at the beginning.
- Select File --> Import --> Solver Deck
- Use the file selector to navigate to the DAT file you wish to open.
- Select Import

Answered: What does a "stiff" system mean?

Mon, 07 Mar 2022 18:00:13 +0000

There are lots of definitions for a stiff system. I am going to collect some I have found useful here:

If the system has widely varying time scales, and the phenomena that change on fast scales are stable, then the problem is stiff [Ascher & Petzold 1998]. Found here, but there is no reference to source given in the citation.
J. D. Lambert defines stiffness as follows: If a numerical method with a finite region of absolute stability, applied to a system with any initial conditions, is forced to use in a certain interval of integration a step length which is excessively small in relation to the smoothness of the exact solution in that interval, then the system is said to be stiff in that interval. Found on Wikipedia here.

If the system is linear:

A linear constant coefficient system is stiff if all of its eigenvalues have negative real part and the stiffness ratio is large. Found on Wikipedia here.

Other notes that might be helpful:

"In some cases there may be two different problems with the same solution, yet one is not stiff and the other is. The phenomenon cannot therefore be a property of the exact solution, since this is the same for both problems, and must be a property of the differential system itself. Such systems are thus known as stiff systems." Found on Wikipedia here.