somtimes the camera will crash

[haoyankai]

[1944301031634F1300] [1.5] [1749613859.732] [host] [warning] Device crashed, but no crash dump could be extracted.
Stack trace (most recent call last):
#30 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#29 Object "python", at 0x5b98cd, in
#28 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9717c9f082, in __libc_start_main
#27 Object "python", at 0x5b9a78, in Py_BytesMain
#26 Object "python", at 0x5f6bdd, in Py_RunMain
#25 Object "python", at 0x5fc0a2, in _PyRun_AnyFileObject
#24 Object "python", at 0x5fc2ee, in _PyRun_SimpleFileObject
#23 Object "python", at 0x5fcd21, in
#22 Object "python", at 0x5e8aef, in
#21 Object "python", at 0x5ec746, in
#20 Object "python", at 0x5ca4ee, in PyEval_EvalCode
#19 Object "python", at 0x5caead, in
#18 Object "python", at 0x516c6b, in _PyEval_EvalFrameDefault
#17 Object "python", at 0x51e130, in PyObject_Vectorcall
#16 Object "python", at 0x556d62, in
#15 Object "python", at 0x55774e, in
#14 Object "python", at 0x514f3c, in _PyEval_EvalFrameDefault
#13 Object "python", at 0x556d11, in
#12 Object "python", at 0x504f03, in _PyObject_MakeTpCall
#11 Object "python", at 0x5423fa, in PyObject_Call
#10 Object "python", at 0x528526, in
#9 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d6dd166f, in
#8 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d6e0a542, in
#7 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d7036a54, in dai::DeviceBase::close()
#6 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d702ff54, in dai::Device::closeImpl()
#5 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d704a6fe, in dai::DeviceBase::closeImpl()
#4 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f96d7036df3, in dai::DeviceBase::~DeviceBase()
#3 Object "/home/wuxin/tools/anaconda3/envs/dexmimicgen/bin/../lib/libstdc++.so.6", at 0x7f96d96333c4, in std::terminate()
#2 Object "/home/wuxin/tools/anaconda3/envs/dexmimicgen/bin/../lib/libstdc++.so.6", at 0x7f96d9633359, in
#1 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9717c9d858, in abort
#0 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9717cbe00b, in gsignal
Aborted (Signal sent by tkill() 721114 1000)
Aborted (core dumped)

[brmarkus]

Can you share more details, please?
Which camera, which firmware version, which operating system, which Python version - which software version, which example/demo/source code?
What is the use case, how to reproduce the crash?

[haoyankai]

thank you for your response
I use two cameras to capture images at the same time,as listed blow:
===Connected to 1844301051CA341300

MXID: 1844301051CA341300
Num of cameras: 3
USB speed: UsbSpeed.SUPER
Board name: DM9098
Product name: OAK-D Pro AF
===Connected to 1944301031634F1300
MXID: 1944301031634F1300
Num of cameras: 3
USB speed: UsbSpeed.SUPER
Board name: DM9095
Product name: OAK-D Lite

operating system:
Linux version 5.15.0-72-generic (buildd@lcy02-amd64-074) (gcc (Ubuntu 9.4.0-1ubuntu120.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #7920.04.1-Ubuntu SMP Thu Apr 20 22:12:07 UTC 2023
python:
3.11.0
depthai version:
2.24.0.0
code:
import argparse

import math

import collections

import numpy as np

import cv2
import depthai as dai
import contextlib
import shutil
import time

import cv2

import json
import os

def createPipeline():

# Start defining a pipeline

pipeline = dai.Pipeline()

# Define a source - color camera

camRgb = pipeline.create(dai.node.ColorCamera)


camRgb.setPreviewSize(300, 300)

camRgb.setBoardSocket(dai.CameraBoardSocket.CAM_A)

camRgb.setResolution(dai.ColorCameraProperties.SensorResolution.THE_1080_P)

camRgb.setInterleaved(False)


# Create output

xoutRgb = pipeline.create(dai.node.XLinkOut)

xoutRgb.setStreamName("rgb")

camRgb.preview.link(xoutRgb.input)


return pipeline

if name == "main":

with contextlib.ExitStack() as stack:
    deviceInfos = dai.Device.getAllAvailableDevices()
    usbSpeed = dai.UsbSpeed.SUPER
    openVinoVersion = dai.OpenVINO.Version.VERSION_2021_4
    qRgbMap = []
    devices = []
    for deviceInfo in deviceInfos:
        deviceInfo: dai.DeviceInfo
        device: dai.Device = stack.enter_context(dai.Device(openVinoVersion, deviceInfo, usbSpeed))
        devices.append(device)
        print("===Connected to ", deviceInfo.getMxId())
        mxId = device.getMxId()
        cameras = device.getConnectedCameras()
        usbSpeed = device.getUsbSpeed()
        eepromData = device.readCalibration2().getEepromData()
        print("   >>> MXID:", mxId)
        print("   >>> Num of cameras:", len(cameras))
        print("   >>> USB speed:", usbSpeed)
        if eepromData.boardName != "":
            print("   >>> Board name:", eepromData.boardName)
        if eepromData.productName != "":
            print("   >>> Product name:", eepromData.productName)

        pipeline = createPipeline()

        device.startPipeline(pipeline)


        # Output queue will be used to get the rgb frames from the output defined above

        q_rgb = device.getOutputQueue(name="rgb", maxSize=4, blocking=False)

        #stream_name = "rgb-" + mxId + "-" + eepromData.productName
        stream_name = mxId
        qRgbMap.append((q_rgb, stream_name))
    
    #wait camera init, wait long to ensure not core
    #time.sleep(8)

    while True:
        for q_rgb, stream_name in qRgbMap:
            if q_rgb.has():
                if stream_name == "1844301051CA341300":
                    image = q_rgb.get().getCvFrame()
                elif stream_name == "1944301031634F1300":
                    wrist_image = q_rgb.get().getCvFrame()
                #cv2.imshow(stream_name, q_rgb.get().getCvFrame())
        if cv2.waitKey(1) == ord('q'):
            break

most of the times it runs ok, but occasionally it will crush,most of the crush happens when the program about to exit, but sometimes it will crush in the running process, I dont know how to deal with it,could you please help me solve it, thank you so much

[haoyankai]

@brmarkus

[haoyankai]

===Connected to 1844301051CA341300

MXID: 1844301051CA341300
Num of cameras: 3
USB speed: UsbSpeed.SUPER
Board name: DM9098
Product name: OAK-D Pro AF
===Connected to 1944301031634F1300
MXID: 1944301031634F1300
Num of cameras: 3
USB speed: UsbSpeed.SUPER
Board name: DM9095
Product name: OAK-D Lite
[1944301031634F1300] [1.5] [1749625661.102] [host] [warning] Device crashed, but no crash dump could be extracted.
Stack trace (most recent call last):
#30 Object "[0xffffffffffffffff]", at 0xffffffffffffffff, in
#29 Object "python", at 0x5b98cd, in
#28 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9fb0dd1082, in __libc_start_main
#27 Object "python", at 0x5b9a78, in Py_BytesMain
#26 Object "python", at 0x5f6bdd, in Py_RunMain
#25 Object "python", at 0x5fc0a2, in _PyRun_AnyFileObject
#24 Object "python", at 0x5fc2ee, in _PyRun_SimpleFileObject
#23 Object "python", at 0x5fcd21, in
#22 Object "python", at 0x5e8aef, in
#21 Object "python", at 0x5ec746, in
#20 Object "python", at 0x5ca4ee, in PyEval_EvalCode
#19 Object "python", at 0x5caead, in
#18 Object "python", at 0x516c6b, in _PyEval_EvalFrameDefault
#17 Object "python", at 0x51e130, in PyObject_Vectorcall
#16 Object "python", at 0x556d62, in
#15 Object "python", at 0x55774e, in
#14 Object "python", at 0x514f3c, in _PyEval_EvalFrameDefault
#13 Object "python", at 0x556d11, in
#12 Object "python", at 0x504f03, in _PyObject_MakeTpCall
#11 Object "python", at 0x5423fa, in PyObject_Call
#10 Object "python", at 0x528526, in
#9 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f6ff0366f, in
#8 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f6ff3c542, in
#7 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f70168a54, in dai::DeviceBase::close()
#6 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f70161f54, in dai::Device::closeImpl()
#5 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f7017c6fe, in dai::DeviceBase::closeImpl()
#4 Object "/home/wuxin/.local/lib/python3.11/site-packages/depthai.cpython-311-x86_64-linux-gnu.so", at 0x7f9f70168df3, in dai::DeviceBase::~DeviceBase()
#3 Object "/home/wuxin/tools/anaconda3/envs/dexmimicgen/bin/../lib/libstdc++.so.6", at 0x7f9f727653c4, in std::terminate()
#2 Object "/home/wuxin/tools/anaconda3/envs/dexmimicgen/bin/../lib/libstdc++.so.6", at 0x7f9f72765359, in
#1 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9fb0dcf858, in abort
#0 Object "/lib/x86_64-linux-gnu/libc.so.6", at 0x7f9fb0df000b, in gsignal
Aborted (Signal sent by tkill() 766125 1000)
Aborted (core dumped)

crash log just happened， in the running process

[brmarkus]

Do you have a chance to run your application in a debugger (like VisualStudioCode debugger), to see the callstack and to check variables in the different frames of the callstack?
In the callstack it's not visible why suddenly "dai::DeviceBase::close()" gets called.
Can you try adding try-catch exception blocks around several code blocks?

In a separate terminal, can you grab the system logs with e.g. "sudo dmesg -w" (watching the log messages), please?
Do you see something happening shortly before or after the crash?
Maybe an unplug of one or both devices gets reported.

How have you connected the devices? Directly to the USB-ports, or using an actively powered USB-hub?

[haoyankai]

thank you for you response, I used a USB extended line, now i remove it and test it again with methods you mentioned to collect some logs