如何在忘乎所以时察觉对方的情感变化

如何在忘乎所以时察觉对方的情感变化

如何在忘乎所以时察觉对方的情感变化

本项目用于对表情进行识别，可以利用表情来察觉情感。

一、项目背景

在学习了微表情心理学后，认识到读懂表情对认识一个人的情感变化的意义，从而诞生了做这一项目的想法。

载入所需库

import paddle
import numpy as np
import cv2
from dels import resnet50
from paddle.vision.datasets import DatasetFolder
import matplotlib.pylab as plt
import os

定义参数

train_file='train'
valid_file='valid'
test_file='test'
imagesize=32
batch_size=32
lr=1e-5

二、数据集介绍

本项目用fer2013数据集，事先数据集已完成对训练集和验证集的切分，同时已将不同表情放于不同文件夹中。具体表情对应的标签和中英文如下：0 anger 生气； 1 disgust 厌恶； 2 fear 恐惧； 3 happy 开心； 4 sad 伤心；5 surprised 惊讶； 6 normal 中性。

1.解压数据集

!unzip -oq /home/aistudio/work/image/fer2013.zip

2.对数据加载进行预处理

# 定义数据预处理
def load_image(img_path):img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)#resizeimg = size(img,(imagesize,imagesize))img = np.array(img).astype('float32')# HWC to CHW img = anspose((2,0,1))#Normalizeimg = img / 255         return img# 构建Dataset
class Face(DatasetFolder):def __init__(self, path):super().__init__(path)def __getitem__(self, index):img_path, label = self.samples[index]label = np.array(label).astype(np.int64)return load_image(img_path), labeltrain_dataset = Face(train_file)
eval_dataset = Face(valid_file)

3.对数据集查看

plt.figure(figsize=(15, 15))
for i in range(5):fundus_img, lab = train_dataset.__getitem__(i)plt.subplot(2, 5, i+1)plt.imshow(anspose(1, 2, 0))plt.axis("off")print(lab)

三、模型选择和开发

选择了resnet50,又用了两个全连接层，然后输出。

1.模型组网

class Layer):def __init__(self):super(Network, self).__init__()snet = resnet50(pretrained=True, num_classes=0)  self.flatten = Flatten()self.linear_1 = Linear(2048, 512)self.linear_2 = Linear(512, 256)self.linear_3 = Linear(256, lu = ReLU()self.dropout = Dropout(0.2)def forward(self, inputs):# print('input', inputs)y = snet(inputs)y = self.flatten(y)y = self.linear_1(y)y = self.linear_2(y)y = lu(y)y = self.dropout(y)y = self.linear_3(y)y = functional.sigmoid(y)return y

2.实例化模型和模型可视化

inputs = paddle.static.InputSpec(shape=[None, 3, 32, 32], name='inputs')
labels = paddle.static.InputSpec(shape=[None, 2], name='labels')model = paddle.Model(Network(), inputs, labels)
paddle.summary(Network(), (1, 3, 32, 32))

-------------------------------------------------------------------------------Layer (type)         Input Shape          Output Shape         Param #    
===============================================================================Conv2D-54        [[1, 3, 32, 32]]     [1, 64, 16, 16]         9,408     BatchNorm2D-54     [[1, 64, 16, 16]]     [1, 64, 16, 16]          256      ReLU-19        [[1, 64, 16, 16]]     [1, 64, 16, 16]           0       MaxPool2D-2      [[1, 64, 16, 16]]      [1, 64, 8, 8]            0       Conv2D-56        [[1, 64, 8, 8]]       [1, 64, 8, 8]          4,096     BatchNorm2D-56      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      ReLU-20         [[1, 256, 8, 8]]      [1, 256, 8, 8]           0       Conv2D-57        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     BatchNorm2D-57      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      Conv2D-58        [[1, 64, 8, 8]]       [1, 256, 8, 8]        16,384     BatchNorm2D-58      [[1, 256, 8, 8]]      [1, 256, 8, 8]         1,024     Conv2D-55        [[1, 64, 8, 8]]       [1, 256, 8, 8]        16,384     BatchNorm2D-55      [[1, 256, 8, 8]]      [1, 256, 8, 8]         1,024     
BottleneckBlock-17    [[1, 64, 8, 8]]       [1, 256, 8, 8]           0       Conv2D-59        [[1, 256, 8, 8]]      [1, 64, 8, 8]         16,384     BatchNorm2D-59      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      ReLU-21         [[1, 256, 8, 8]]      [1, 256, 8, 8]           0       Conv2D-60        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     BatchNorm2D-60      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      Conv2D-61        [[1, 64, 8, 8]]       [1, 256, 8, 8]        16,384     BatchNorm2D-61      [[1, 256, 8, 8]]      [1, 256, 8, 8]         1,024     
BottleneckBlock-18    [[1, 256, 8, 8]]      [1, 256, 8, 8]           0       Conv2D-62        [[1, 256, 8, 8]]      [1, 64, 8, 8]         16,384     BatchNorm2D-62      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      ReLU-22         [[1, 256, 8, 8]]      [1, 256, 8, 8]           0       Conv2D-63        [[1, 64, 8, 8]]       [1, 64, 8, 8]         36,864     BatchNorm2D-63      [[1, 64, 8, 8]]       [1, 64, 8, 8]           256      Conv2D-64        [[1, 64, 8, 8]]       [1, 256, 8, 8]        16,384     BatchNorm2D-64      [[1, 256, 8, 8]]      [1, 256, 8, 8]         1,024     
BottleneckBlock-19    [[1, 256, 8, 8]]      [1, 256, 8, 8]           0       Conv2D-66        [[1, 256, 8, 8]]      [1, 128, 8, 8]        32,768     BatchNorm2D-66      [[1, 128, 8, 8]]      [1, 128, 8, 8]          512      ReLU-23         [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-67        [[1, 128, 8, 8]]      [1, 128, 4, 4]        147,456    BatchNorm2D-67      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      Conv2D-68        [[1, 128, 4, 4]]      [1, 512, 4, 4]        65,536     BatchNorm2D-68      [[1, 512, 4, 4]]      [1, 512, 4, 4]         2,048     Conv2D-65        [[1, 256, 8, 8]]      [1, 512, 4, 4]        131,072    BatchNorm2D-65      [[1, 512, 4, 4]]      [1, 512, 4, 4]         2,048     
BottleneckBlock-20    [[1, 256, 8, 8]]      [1, 512, 4, 4]           0       Conv2D-69        [[1, 512, 4, 4]]      [1, 128, 4, 4]        65,536     BatchNorm2D-69      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      ReLU-24         [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-70        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    BatchNorm2D-70      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      Conv2D-71        [[1, 128, 4, 4]]      [1, 512, 4, 4]        65,536     BatchNorm2D-71      [[1, 512, 4, 4]]      [1, 512, 4, 4]         2,048     
BottleneckBlock-21    [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-72        [[1, 512, 4, 4]]      [1, 128, 4, 4]        65,536     BatchNorm2D-72      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      ReLU-25         [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-73        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    BatchNorm2D-73      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      Conv2D-74        [[1, 128, 4, 4]]      [1, 512, 4, 4]        65,536     BatchNorm2D-74      [[1, 512, 4, 4]]      [1, 512, 4, 4]         2,048     
BottleneckBlock-22    [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-75        [[1, 512, 4, 4]]      [1, 128, 4, 4]        65,536     BatchNorm2D-75      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      ReLU-26         [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-76        [[1, 128, 4, 4]]      [1, 128, 4, 4]        147,456    BatchNorm2D-76      [[1, 128, 4, 4]]      [1, 128, 4, 4]          512      Conv2D-77        [[1, 128, 4, 4]]      [1, 512, 4, 4]        65,536     BatchNorm2D-77      [[1, 512, 4, 4]]      [1, 512, 4, 4]         2,048     
BottleneckBlock-23    [[1, 512, 4, 4]]      [1, 512, 4, 4]           0       Conv2D-79        [[1, 512, 4, 4]]      [1, 256, 4, 4]        131,072    BatchNorm2D-79      [[1, 256, 4, 4]]      [1, 256, 4, 4]         1,024     ReLU-27        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-80        [[1, 256, 4, 4]]      [1, 256, 2, 2]        589,824    BatchNorm2D-80      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-81        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-81     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     Conv2D-78        [[1, 512, 4, 4]]     [1, 1024, 2, 2]        524,288    BatchNorm2D-78     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-24    [[1, 512, 4, 4]]     [1, 1024, 2, 2]           0       Conv2D-82       [[1, 1024, 2, 2]]      [1, 256, 2, 2]        262,144    BatchNorm2D-82      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     ReLU-28        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-83        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    BatchNorm2D-83      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-84        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-84     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-25   [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-85       [[1, 1024, 2, 2]]      [1, 256, 2, 2]        262,144    BatchNorm2D-85      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     ReLU-29        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-86        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    BatchNorm2D-86      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-87        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-87     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-26   [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-88       [[1, 1024, 2, 2]]      [1, 256, 2, 2]        262,144    BatchNorm2D-88      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     ReLU-30        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-89        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    BatchNorm2D-89      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-90        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-90     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-27   [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-91       [[1, 1024, 2, 2]]      [1, 256, 2, 2]        262,144    BatchNorm2D-91      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     ReLU-31        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-92        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    BatchNorm2D-92      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-93        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-93     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-28   [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-94       [[1, 1024, 2, 2]]      [1, 256, 2, 2]        262,144    BatchNorm2D-94      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     ReLU-32        [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-95        [[1, 256, 2, 2]]      [1, 256, 2, 2]        589,824    BatchNorm2D-95      [[1, 256, 2, 2]]      [1, 256, 2, 2]         1,024     Conv2D-96        [[1, 256, 2, 2]]     [1, 1024, 2, 2]        262,144    BatchNorm2D-96     [[1, 1024, 2, 2]]     [1, 1024, 2, 2]         4,096     
BottleneckBlock-29   [[1, 1024, 2, 2]]     [1, 1024, 2, 2]           0       Conv2D-98       [[1, 1024, 2, 2]]      [1, 512, 2, 2]        524,288    BatchNorm2D-98      [[1, 512, 2, 2]]      [1, 512, 2, 2]         2,048     ReLU-33        [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       Conv2D-99        [[1, 512, 2, 2]]      [1, 512, 1, 1]       2,359,296   BatchNorm2D-99      [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     Conv2D-100        [[1, 512, 1, 1]]     [1, 2048, 1, 1]       1,048,576   BatchNorm2D-100    [[1, 2048, 1, 1]]     [1, 2048, 1, 1]         8,192     Conv2D-97       [[1, 1024, 2, 2]]     [1, 2048, 1, 1]       2,097,152   BatchNorm2D-97     [[1, 2048, 1, 1]]     [1, 2048, 1, 1]         8,192     
BottleneckBlock-30   [[1, 1024, 2, 2]]     [1, 2048, 1, 1]           0       Conv2D-101       [[1, 2048, 1, 1]]      [1, 512, 1, 1]       1,048,576   BatchNorm2D-101     [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     ReLU-34        [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       Conv2D-102        [[1, 512, 1, 1]]      [1, 512, 1, 1]       2,359,296   BatchNorm2D-102     [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     Conv2D-103        [[1, 512, 1, 1]]     [1, 2048, 1, 1]       1,048,576   BatchNorm2D-103    [[1, 2048, 1, 1]]     [1, 2048, 1, 1]         8,192     
BottleneckBlock-31   [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       Conv2D-104       [[1, 2048, 1, 1]]      [1, 512, 1, 1]       1,048,576   BatchNorm2D-104     [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     ReLU-35        [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       Conv2D-105        [[1, 512, 1, 1]]      [1, 512, 1, 1]       2,359,296   BatchNorm2D-105     [[1, 512, 1, 1]]      [1, 512, 1, 1]         2,048     Conv2D-106        [[1, 512, 1, 1]]     [1, 2048, 1, 1]       1,048,576   BatchNorm2D-106    [[1, 2048, 1, 1]]     [1, 2048, 1, 1]         8,192     
BottleneckBlock-32   [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       
AdaptiveAvgPool2D-2  [[1, 2048, 1, 1]]     [1, 2048, 1, 1]           0       ResNet-2         [[1, 3, 32, 32]]     [1, 2048, 1, 1]           0       Flatten-2       [[1, 2048, 1, 1]]        [1, 2048]              0       Linear-4           [[1, 2048]]            [1, 512]          1,049,088   Linear-5            [[1, 512]]            [1, 256]           131,328    ReLU-36            [[1, 256]]            [1, 256]              0       Dropout-2           [[1, 256]]            [1, 256]              0       Linear-6            [[1, 256]]             [1, 8]             2,056     
===============================================================================
Total params: 24,743,624
Trainable params: 24,637,384
Non-trainable params: 106,240
-------------------------------------------------------------------------------
Input size (MB): 0.01
Forward/backward pass size (MB): 5.39
Params size (MB): 94.39
Estimated Total Size (MB): 99.79
-------------------------------------------------------------------------------
{'total_params': 24743624, 'trainable_params': 24637384}

3.模型训练

# 模型训练相关配置，准备损失计算方法，优化器和精度计算方法
model.prepare(paddle.optimizer.Adam(learning_rate=lr, parameters=model.parameters()),CrossEntropyLoss(),ic.Accuracy())# 模型训练model.fit(train_data=train_dataset,     #训练数据集eval_data=eval_dataset,         #测试数据集batch_size=batch_size,          #一个批次的样本数量epochs=27,                      #迭代轮次save_dir="/home/aistudio/lup", #把模型参数、优化器参数保存至自定义的文件夹save_freq=3,                  #设定每隔多少个epoch保存模型参数及优化器参数 verbose=1                )

step 898/898 [==============================] - loss: 1.8075 - acc: 0.2449 - 53ms/step         
save checkpoint at /home/aistudio/lup/0

step 113/113 [==============================] - loss: 1.7850 - acc: 0.2530 - 23ms/step        
Eval samples: 3589

4.模型评估测试

model.evaluate(eval_dataset, batch_size=5, verbose=1)

step 718/718 [==============================] - loss: 1.9499 - acc: 0.4870 - 22ms/step         
Eval samples: 3589
{'loss': [1.9498544], 'acc': 0.48704374477570356}

四、预测

1.对预测数据处理

def load_test(img_path):   img = cv2.cvtColor(cv2.imread(img_path), cv2.COLOR_BGR2RGB)#resizeimg = size(img,(imagesize,imagesize))img = np.array(img).astype('float32')# HWC to CHW img = anspose((2,0,1))img = np.expand_dims(img, axis=0)#Normalizeimg = img / 255         return imgtest_dataset=[]
for i in os.listdir(test_file):test_dataset.append(load_test(test_file+'//'+i))

2.对模型预测

# 进行预测操作
result = model.predict(test_dataset)# 定义产出数字与表情的对应关系
face={0:'anger',1:'disgust',2:'fear',3:'happy',4:'sad',5:'surprised',6:'normal'}# 定义画图方法
def show_img(img, predict):plt.figure()plt.title('predict: {}'.format(face[predict]))plt.shape([3, 32, 32]).transpose(1,2,0))plt.show()# 抽样展示
indexs = [4, 15, 45,]for idx in indexs:show_img(test_dataset[idx][0], np.argmax(result[0][idx]))

step 3589/3589 [==============================] - 21ms/step         
Predict samples: 3589

五、效果展示

五、总结与升华

1.在图像分类中输出数要大于所分图像类数。

2.对于维度缺失问题可以用img = np.expand_dims(img, axis=0)增加维度。

六、个人介绍

太原理工大学 软件学院 软件工程专业 2020级 本科生 王志洲

AIstudio地址链接：

码云地址链接：