environment

build

cd /opt/qcom/Hexagon_SDK
ln -s 5.5.5.0 HexagonSDK

export QNN_SDK_ROOT=/opt/qcom/aistack/qairt/2.31.0.250130
source $QNN_SDK_ROOT/bin/envsetup.sh
source /opt/qcom/Hexagon_SDK/5.5.5.0/setup_sdk_env.source

git clone https://github.com/UbiquitousLearning/mllm.git

CMakeLists.txt 中gtest相关的注释掉



cd src/backends/qnn
ln -s $QNN_SDK_ROOT sdk

cd src/backends/qnn/LLaMAOpPackageHtp/LLaMAPackage/
make htp_aarch64 && make htp_v75

./build_android_qnn.sh
./run_qwen_qnn.sh

要加下这两句:
adb shell mkdir -p /data/local/tmp/mllm/bin
adb push ../vocab/qwen_merges.txt /data/local/tmp/mllm/vocab/

result

./run_qwen_qnn.sh 
../vocab/qwen_vocab.mllm: 1 file pushed, 0 skipped. 95.0 MB/s (3189278 bytes in 0.032s)
../vocab/qwen_merges.txt: 1 file pushed, 0 skipped. 81.8 MB/s (1671839 bytes in 0.019s)
qwen-1.5-1.8b-chat-int8 file already exists
qwen-1.5-1.8b-chat-q4k.mllm file already exists
QNN_SDK_ROOT is set to /opt/qcom/aistack/qairt/2.31.0.250130
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtp.so: 1 file pushed, 0 skipped. 53.6 MB/s (2060792 bytes in 0.037s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtpV75Stub.so: 1 file pushed, 0 skipped. 22.9 MB/s (453640 bytes in 0.019s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtpPrepare.so: 1 file pushed, 0 skipped. 33.4 MB/s (55080032 bytes in 1.575s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtpProfilingReader.so: 1 file pushed, 0 skipped. 113.0 MB/s (579072 bytes in 0.005s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtpOptraceProfilingReader.so: 1 file pushed, 0 skipped. 153.1 MB/s (3073968 bytes in 0.019s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/aarch64-android/libQnnHtpV75CalculatorStub.so: 1 file pushed, 0 skipped. 0.4 MB/s (6472 bytes in 0.015s)
/opt/qcom/aistack/qairt/2.31.0.250130/lib/hexagon-v75/unsigned/libQnnHtpV75Skel.so: 1 file pushed, 0 skipped. 89.1 MB/s (8485728 bytes in 0.091s)
../src/backends/qnn/LLaMAOpPackageHtp/LLaMAPackage/build/aarch64-android/libQnnLLaMAPackage.so: 1 file pushed, 0 skipped. 300.3 MB/s (1479704 bytes in 0.005s)
../src/backends/qnn/LLaMAOpPackageHtp/LLaMAPackage/build/hexagon-v75/libQnnLLaMAPackage.so: 1 file pushed, 0 skipped. 177.0 MB/s (818800 bytes in 0.004s)
../bin-arm/demo_qwen_npu: 1 file pushed, 0 skipped. 72.1 MB/s (93400608 bytes in 1.235s)
[INFO] Sun Jul 13 14:43:12 2025 [/home/liuqi/code/llm/genie/mllm/src/backends/qnn/QNNBackend.cpp:118] Backend: libQnnHtp.so
[INFO] Sun Jul 13 14:43:12 2025 [/home/liuqi/code/llm/genie/mllm/src/backends/qnn/QNNBackend.cpp:143] Backend        build version: v2.31.0.250130151446_114721
[INFO] Sun Jul 13 14:43:12 2025 [/home/liuqi/code/llm/genie/mllm/src/backends/qnn/QNNBackend.cpp:166] Initialize Backend Returned Status = 0
[INFO] Sun Jul 13 14:43:12 2025 [/home/liuqi/code/llm/genie/mllm/src/backends/qnn/QNNBackend.cpp:636] Registered Op Package: libQnnLLaMAPackage_CPU.so and interface provider: LLaMAPackageInterfaceProvider
[INFO] Sun Jul 13 14:43:12 2025 [/home/liuqi/code/llm/genie/mllm/src/backends/qnn/QNNBackend.cpp:636] Registered Op Package: libQnnLLaMAPackage_HTP.so and interface provider: LLaMAPackageInterfaceProvider
Warmup finished.
[Q] "Large Language Models (LLMs) are advanced artificial intelligence systems designed to understand and generate human-like text. These models are trained on vast amounts of data, enabling them to perform a wide range of tasks, from answering questions and summarizing text to generating creative content and engaging in conversational dialogue. LLMs like GPT-3 and GPT-4, developed by OpenAI, have set new benchmarks in natural language processing by leveraging deep learning architectures, particularly transformer models, which excel at capturing context and relationships within text. The scalability and versatility of LLMs make them invaluable tools for applications in education, customer service, content creation, and more. However, their deployment also raises ethical considerations, including issues of bias, misinformation, and the potential for misuse. As the field continues to evolve, ongoing research and responsible deployment strategies are essential to harnessing the full potential of these powerful AI systems while mitigating their risks."
Generate a title based on the above text.
[A] Exploring the Advantages and Ethical Considerations of Large Language Models (LLMs): A Comprehensive OverviewSegmentation fault

issue

references