YOLO v2を試す

YOLOの解説

darkflowを使った
https://github.com/thtrieu/darkflow
YOLO v2のcfgとweightsのダウンロード
https://pjreddie.com/darknet/yolo/
darkflowのインストール

python3 setup.py build_ext --inplace

実行

./flow --model cfg/yolo-voc.cfg --load bin/yolo-voc.weights --demo hoge.avi --saveVideo

実行結果

ゲームx強化学習

Introduction to AI for Video Games (Reinforcement Learning)

音楽生成

PYNQでHDMI＋OpenCV

PYNQはv2.0だとpynq.driversがないようなのでHDMIが使えない（追記：ｖ2.0のドキュメントに気づかなかった。APIが多少変わっているが普通に動きます。）
v1.4を使う
https://github.com/Xilinx/PYNQ/tree/v1.4
face detect sample
https://github.com/Xilinx/PYNQ/blob/v1.4/Pynq-Z1/notebooks/examples/opencv_face_detect_hdmi.ipynb

PYNQのubuntu 15.10 (wily)のaptのsources.list

/etc/apt/sources.list.d/multistrap-wily.list

#以下を変更する
#deb [arch=armhf] http://ports.ubuntu.com/ubuntu-ports wily universe main
#deb-src http://ports.ubuntu.com/ubuntu-ports wily universe main
deb [arch=armhf] http://old-releases.ubuntu.com/ubuntu/ wily universe main
deb-src http://old-releases.ubuntu.com/ubuntu/ wily universe main

$ sudo apt-get update

Ubuntuのaptのsources.listの書き方

http://blog.goo.ne.jp/j_adversaria/e/4c024665dc61a79510d72cdf5444467a

書式：deb URL Distribution [component1] [component2]...
書式：deb-src URL Distribution [component1] [component2]...

PYNQにCaffeをインストール

sudo apt-get install -y libgflags-dev libgtest-dev libc++-dev clang libgoogle-glog-dev liblmdb-dev

PYNQでリアルタイムエッジ検出

canny.py

from pynq import Overlay
Overlay("base.bit").download()
import time
import cv2
import numpy as np
from pynq.drivers import HDMI
from pynq.drivers.video import VMODE_1920x1080
hdmi_out = HDMI('out', video_mode=VMODE_1920x1080)
hdmi_in = HDMI('in', init_timeout=10, frame_list=hdmi_out.frame_list)
hdmi_in.start()
hdmi_out.start()

for i in range(10):
	print("waiting ",10-i)
	time.sleep(1)

print("change HDMI output frame")
hdmi_out.frame_index_next()
print("main loop")

try:
  while True:
    frame = hdmi_in.frame_raw()
    np_frame= (np.frombuffer(frame, dtype=np.uint8)).reshape(1080,1920,3)
    edges = cv2.Canny(np_frame,100,200)
    stacked_img = np.stack((edges,)*3,axis=-1)
    hdmi_out.frame_raw(bytearray(stacked_img.tobytes()))

except KeyboardInterrupt:
    hdmi_out.stop()
    hdmi_in.stop()
    del hdmi_in, hdmi_out

実行

$ sudo python3.6 canny.py

PYNQでface detection

facedetect.py

from pynq import Overlay
Overlay("base.bit").download()
import time
import cv2
import numpy as np
from pynq.drivers import HDMI
from pynq.drivers.video import VMODE_1920x1080
hdmi_out = HDMI('out', video_mode=VMODE_1920x1080)
hdmi_in = HDMI('in', init_timeout=10, frame_list=hdmi_out.frame_list)
hdmi_in.start()
hdmi_out.start()

for i in range(10):
	print("waiting ",10-i)
	time.sleep(1)

print("change HDMI output frame")
hdmi_out.frame_index_next()
print("main loop")

try:
  while True:
    frame = hdmi_in.frame_raw()
    np_frame= (np.frombuffer(frame, dtype=np.uint8)).reshape(1080,1920,3)
    face_cascade = cv2.CascadeClassifier(
                        './data/haarcascade_frontalface_default.xml')
    eye_cascade = cv2.CascadeClassifier(
                        './data/haarcascade_eye.xml')
    gray = cv2.cvtColor(np_frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3, 5)
    for (x,y,w,h) in faces:
        cv2.rectangle(np_frame,(x,y),(x+w,y+h),(255,0,0),2)
        roi_gray = gray[y:y+h, x:x+w]
        roi_color = np_frame[y:y+h, x:x+w]
        eyes = eye_cascade.detectMultiScale(roi_gray)
        for (ex,ey,ew,eh) in eyes:
            cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
    hdmi_out.frame_raw(bytearray(np_frame.tobytes()))

except KeyboardInterrupt:
    print("terminating application")
    hdmi_out.stop()
    hdmi_in.stop()
    del hdmi_in, hdmi_out