試Raspberry Pi 5 + Hailo AI (同時試 Orange Pi 5, Python AI programming)

Rank: 3 Rank: 3 Rank: 3

2^#

發表於 2025-1-15 15:04 | 只看該作者

我自己剛剛試 pull model, 無意外, 係唔夠 memory.

買了16gb 後再嚟.

chingkit

Rank: 3 Rank: 3 Rank: 3

3^#

發表於 2025-1-15 23:00 | 只看該作者

就算有16GB RAM，行LLM足夠快嗎？

via HKEPC IR 5.1.14 - Android(5.1.2F)

sk2k

Rank: 3 Rank: 3 Rank: 3

4^#

發表於 2025-1-16 00:36 | 只看該作者

得8G 咁大貪行 14B 緊係唔夠....

Rank: 3 Rank: 3 Rank: 3

5^#

發表於 2025-1-16 04:57 | 只看該作者

要行到 AI on Ras Pi, 係要有 Hailo8l 13Tops:

The Hailo-8L Entry-Level AI Accelerator, featuring up to 13 tera-operations per second (TOPS), is designed to support entry level products requiring limited AI capacity or lower performance. It is competitively priced and features superior area and power efficiency compared to category leading alternatives. The Hailo-8L AI accelerator boasts exceptional low-latency, high-efficiency processing, capable of handling complex pipelines with multiple real-time streams and concurrent processing of multiple models and AI tasks. It is compatible with the Hailo-8 field-proven and comprehensive software suite for seamless future upgrade to higher AI capacities.

The AI HAT+ is available in 13 and 26 tera-operations per second (TOPS) variants,
built around the Hailo-8L and Hailo-8 neural network inference accelerators. The 13
TOPS variant capably runs neural networks for applications including object detection,
semantic and instance segmentation, pose estimation, and more. The 26 TOPS variant
can handle larger networks, can run them more quickly, and can more effectively run
multiple networks simultaneously.

Rank: 3 Rank: 3 Rank: 3

6^#

發表於 2025-1-16 05:15 | 只看該作者

run 較細嘅 models 無問題, performance is good. 之後試做自己 AI project (寫 app 用自己 local SLM, train model, AI enhance video etc)

用 16gb 當然最好, 已 order, 收到再講.

Rank: 3 Rank: 3 Rank: 3

7^#

發表於 2025-1-26 14:20 | 只看該作者

一路都知 Orange Pi 係勁嘢, 但怕大6嘢, 易壞, 有後門, 少 community ..., 一直都唔敢點. 依家 Pi 5 行 llm 要快, 要等 Hailo-10H 先得. 唔知幾時出, 幾錢, 掂唔掂, 咁我一向都係想到就即刻做, 又唔貴, 都買咗部 Orange Pi 5 Max 16gb, 等緊. 期望比 Pi 5 要快得多. Pi 5 改為做寫 AI program server.

caihk

Rank: 3 Rank: 3 Rank: 3

8^#

發表於 2025-1-29 10:22 | 只看該作者

呢個帖好值得follow，我用咗rpi好多年，只係用過兩代，rpi3同rpi4。對於日常應用，部rpi4 4G對於我嚟講夠用。前幾月試過用嚟玩whisper，語音轉文字，唔記得用small定係base，速度真係慢，唔多掂，跟住冇再玩。

一直都有留意rpi既野，知道佢有出ai 加速器 halio，同埋再近有出 rpi5 16G，但聽講halio support既野未係好多。另外，性價比方面，似乎香橙派唔錯，仲有啱出既nano jetson super，大約港紙2千，因為係nvidia出，可以玩到好多ai既野，不過就得8G，大些少既model玩唔到。依家只係用另一部舊既J4105+24Gram去試ai啲野，早兩日local部署codeformer做影象修復upscaling，用到不過冇裝獨立顯卡，cpu only都係慢。

香港用sbc玩ai既帖子真係唔多，希望你多啲分享使用心得。不過hkepc好容易關帖，如果你有其他平台交流就更好。