General troubleshooting and problem solving

This guide is for both Dash Enterprise Single Server and Dash Enterprise Kubernetes. For problems that are specific to Dash Enterprise Kubernetes:

Dash Enterprise Kubernetes

LDAP: Testing authentication

You can troubleshoot LDAP authentication by running the ldapsearch tool on your Dash Enterprise server. If you installed Dash Enterprise following the Installation on a Cloud Provider instructions, this tool has already been installed, otherwise you can install it as below:

• Ubuntu: sudo apt install ldap-utils

• RHEL or CentOS: yum install openldap-clients

ldapsearch syntax can be complicated, so here is an example to get you started:

ldapsearch -H ldap://ad.plot.ly -x -D 'plotly\mrplot' -w 'ldappass' -b  'CN=Users,DC=ad,DC=plot,DC=ly' SAMAccountName -v

Breaking it down:

• ldap://ad.plot.ly is the server URI

  • This is the the auth_ldap_server_uri parameter

• plotly\mrplot is the bind DN used for authentication

  • This is the auth_ldap_bind_dn parameter

  • mrplot is the username

  • Your LDAP server may or may not need you to include the LDAP domain and backlash (plotly in this example); check with your server administrator

• ldappass is the password for the bind DN

  • This is the auth_ldap_bind_password parameter

    CN=Users,DC=ad,DC=plot,DC=ly is the *search DN

  • This is the auth_ldap_search_dn parameter

  • This is usually based on the domain name of your LDAP server

The above command returns a list of all accounts in the search DN; in this case, the list of all usernames allowed to log in to Dash Enterprise.

If you receive any other output, contact your LDAP server administrator or Plotly support for help.

Testing without adding entries to your DNS server

You can test your Dash Enterprise instance locally by adding the DNS entries directly to the hosts file on each machine intended to access the instance. Here are instructions on how to do that for various operating systems. The line to add will look something like this:

12.34.56.78 plotly.your-company.com

• Replace 12.34.56.78 with the IP of your Dash Enterprise server

• plotly.your-company.com is the Base Domain value that you specified during configuration

If you set your server’s current hostname as the Base Domain in your Dash Enterprise configuration, you do not need to modify your hosts file.

Private key/certificate debugging

If you suspect your SSL certificate and key do not match, you can verify by finding the sha256sum of each as follows:

openssl x509 -in /path/to/your/plotly_certificate.crt -pubkey -noout -outform pem | sha256sum

openssl pkey -in /path/to/your/plotly_key.key -pubout -outform pem | sha256sum

If the results of the above commands match, then the key and certificate match as well.

Running devicemapper in loopback mode

If your system defaults to using devicemapper in loopback mode (typical for RHEL 7 and CentOS 7), you will see an error during installation that looks like this:

The running Docker daemon is configured to use the 'devicemapper' storage driver in loopback mode.
This is not recommended for production use. Please see the following URL for more information.

https://help.replicated.com/docs/kb/developer-resources/devicemapper-warning/.

Do you want to proceed anyway?

To solve this, cancel installation and configure your system to use direct-lvm according to this Docker guide.

App manager boot failure with max depth exceeded error response from daemon

During startup, you may encounter an error that corresponds with the following message in your Dash container logs:

Error response from daemon: max depth exceeded

You can resolve this as follows:

1 - Find the Dash container ID: sudo docker ps

2 - Stop the Dash container: sudo docker stop DASH_CONTAINER_ID

3 - Delete all Dash-related containers/images:

sudo docker rm -f $(sudo docker ps -a | grep -E "web.1 | Exited" | awk "{print $ 1}")

sudo docker rmi -f $(sudo docker images -a | grep -E "<none> | dokku" | awk "{print $ 3}")

4 - In the Dash Enterprise Server Manager, click Start Now

5 - Redeploy your Dash application

Plotly data directory verification failure

If attempting to set the Plotly data directory and the verification test is failing:

1 - Confirm the directory exists on the system and you have correctly entered the path to it

2 - Ensure SELinux is enabled and running with sudo sestatus, and enable it if not

3 - If SELinux is enabled, install a custom SELinux policy to allow some necessary permissions.

4 - If SELinux is enabled, add SELinux permissions to your data directory with:

sudo chcon -Rt svirt_sandbox_file_t /path/to/your_data_directory

'Permission denied' when accessing mapped directories in apps

If you are running with SELinux enforcing and dash apps give a "Permission denied" error when accessing files on a mapped directory, see Adding SELinux permissions to mapped directories.

Missing container-selinux dependency (Error: nothing to do)

In RHEL, the container-selinux dependency may be missing from the system, which commonly results in the following error when running the installation script:

Error: nothing to do.

To solve this, we recommend the following options:

• Enable the appropriate repository:

  sudo subscription-manager repos --enable=rhel-7-server-extras-rpms

• If the above doesn’t work, install the package directly:

  yum install http://mirror.centos.org/centos/7/extras/x86_64/Packages/container-selinux-2.36-1.gitff95335.el7.noarch.rpm

OCI runtime error

On RHEL < v7.4, OCI runtime issues have been known to occur, preventing Server Manager containers (names beginning with replicated) from starting when using Docker >= 17.12.1. You can resolve this by upgrading to >= RHEL v7.4.

Admin console password reset

When setting up your Server Manager for the first time, you set a password for the admin console. If you need to reset this:

1. Use the command replicatedctl console-auth reset to remove the current password.

2. Go to https://<your server>:8800/create-password to create a new password.

Support

Our support team is available to help if you run into any issues with your installation. When contacting us, please also include a support bundle, which is a collection of log files and app state that will help us diagnose your problem. There are two methods available for creating and sending one:

Directly upload support bundle to our portal (recommended)

1 - SSH into the server running Dash Enterprise

2 - Copy and paste the following command, which generates a bundle and prompts you to upload it to our support portal:

docker pull replicated/support-bundle && \
docker run --rm -it --name support-bundle \
  -v $PWD:/out -w /out \
  -v /var/run/docker.sock:/var/run/docker.sock \
  --net host --pid host  \
  -e HTTP_PROXY -e HTTPS_PROXY -e NO_PROXY \
  replicated/support-bundle generate \
    --channel-id 2981105c67e98b19d315128c52286160

3 - E-mail Plotly support with a description of the issue and let us know you have made a support bundle available for analysis

Download and e-mail support bundle (for airgapped instances)

1 - Browse to your Dash Enterprise Server Manager

2 - Click the Support tab

3 - Click Download Support Bundle

4 - Upload the resulting file to a cloud storage service of your choice

5 - E-mail us with a description of the issue and a link to the support bundle