卷积神经网络机器学习外文文献翻译中英文2020

2024年2月7日发(作者：)

卷积神经网络机器学习相关外文翻译中英文英文2020

Prediction of composite microstructure stress-strain curves using

convolutional neural networks

Charles Yang，Youngsoo Kim，Seunghwa Ryu，Grace Gu

Abstract

Stress-strain curves are an important representation of a material's

mechanical properties, from which important properties such as elastic

modulus, strength, and toughness, are defined. However, generating

stress-strain curves from numerical methods such as finite element

method (FEM) is computationally intensive, especially when considering

the entire failure path for a material. As a result, it is difficult to perform

high throughput computational design of materials with large design

spaces, especially when considering mechanical responses beyond the

elastic limit. In this work, a combination of principal component analysis

(PCA) and convolutional neural networks (CNN) are used to predict the

entire stress-strain behavior of binary composites evaluated over the

entire failure path, motivated by the significantly faster inference speed of

empirical models. We show that PCA transforms the stress-strain curves

into an effective latent space by visualizing the eigenbasis of PCA.

Despite having a dataset of only 10-27% of possible microstructure

configurations, the mean absolute error of the prediction is <10% of the

1

range of values in the dataset, when measuring model performance based

on derived material descriptors, such as modulus, strength, and toughness.

Our study demonstrates the potential to use machine learning to

accelerate material design, characterization, and optimization.

Keywords：Machine learning，Convolutional neural networks，Mechanical properties，Microstructure，Computational mechanics

Introduction

Understanding the relationship between structure and property for

materials is a seminal problem in material science, with significant

applications for designing next-generation materials. A primary

motivating example is designing composite microstructures for

load-bearing applications, as composites offer advantageously high

specific strength and specific toughness. Recent advancements in additive

manufacturing have facilitated the fabrication of complex composite

structures, and as a result, a variety of complex designs have been

fabricated and tested via 3D-printing methods. While more advanced

manufacturing techniques are opening up unprecedented opportunities for

advanced materials and novel functionalities, identifying microstructures

with desirable properties is a difficult optimization problem.

One method of identifying optimal composite designs is by

constructing analytical theories. For conventional

particulate/fiber-reinforced composites, a variety of homogenization

2

theories have been developed to predict the mechanical properties of

composites as a function of volume fraction, aspect ratio, and orientation

distribution of reinforcements. Because many natural composites,

synthesized via self-assembly processes, have relatively periodic and

regular structures, their mechanical properties can be predicted if the load

transfer mechanism of a representative unit cell and the role of the

self-similar hierarchical structure are understood. However, the

applicability of analytical theories is limited in quantitatively predicting

composite properties beyond the elastic limit in the presence of defects,

because such theories rely on the concept of representative volume

element (RVE), a statistical representation of material properties, whereas

the strength and failure is determined by the weakest defect in the entire

sample domain. Numerical modeling based on finite element methods

(FEM) can complement analytical methods for predicting inelastic

properties such as strength and toughness modulus (referred to as

toughness, hereafter) which can only be obtained from full stress-strain

curves.

However, numerical schemes capable of modeling the initiation and

propagation of the curvilinear cracks, such as the crack phase field model,

are computationally expensive and time-consuming because a very fine

mesh is required to accommodate highly concentrated stress field near

crack tip and the rapid variation of damage parameter near diffusive crack

3

surface. Meanwhile, analytical models require significant human effort

and domain expertise and fail to generalize to similar domain problems.

In order to identify high-performing composites in the midst of large

design spaces within realistic time-frames, we need models that can

rapidly describe the mechanical properties of complex systems and be

generalized easily to analogous systems. Machine learning offers the

benefit of extremely fast inference times and requires only training data to

learn relationships between inputs and , composite

microstructures and their mechanical properties. Machine learning has

already been applied to speed up the optimization of several different

physical systems, including graphene kirigami cuts, fine-tuning spin qubit

parameters, and probe microscopy tuning. Such models do not require

significant human intervention or knowledge, learn relationships

efficiently relative to the input design space, and can be generalized to

different systems.

In this paper, we utilize a combination of principal component

analysis (PCA) and convolutional neural networks (CNN) to predict the

entire stress-strain curve of composite failures beyond the elastic limit.

Stress-strain curves are chosen as the model's target because they are

difficult to predict given their high dimensionality. In addition,

stress-strain curves are used to derive important material descriptors such

as modulus, strength, and toughness. In this sense, predicting stress-strain

4

curves is a more general description of composites properties than any

combination of scaler material descriptors. A dataset of 100,000 different

composite microstructures and their corresponding stress-strain curves are

used to train and evaluate model performance. Due to the high

dimensionality of the stress-strain dataset, several dimensionality

reduction methods are used, including PCA, featuring a blend of domain

understanding and traditional machine learning, to simplify the problem

without loss of generality for the model.

We will first describe our modeling methodology and the parameters

of our finite-element method (FEM) used to generate data. Visualizations

of the learned PCA latent space are then presented, along with model

performance results.

CNN implementation and training

A convolutional neural network was trained to predict this lower

dimensional representation of the stress vector. The input to the CNN was

a binary matrix representing the composite design, with 0's corresponding

to soft blocks and 1's corresponding to stiff blocks. PCA was

implemented with the open-source Python package scikit-learn, using the

default hyperparameters. CNN was implemented using Keras with a

TensorFlow backend. The batch size for all experiments was set to 16 and

the number of epochs to 30; the Adam optimizer was used to update the

CNN weights during backpropagation.

5

A train/test split ratio of 95:5 is used –we justify using a smaller

ratio than the standard 80:20 because of a relatively large dataset. With a

ratio of 95:5 and a dataset with 100,000 instances, the test set size still has

enough data points, roughly several thousands, for its results to generalize.

Each column of the target PCA-representation was normalized to have a

mean of 0 and a standard deviation of 1 to prevent instable training.

Finite element method data generation

FEM was used to generate training data for the CNN model.

Although initially obtained training data is compute-intensive, it takes

much less time to train the CNN model and even less time to make

high-throughput inferences over thousands of new, randomly generated

composites. The crack phase field solver was based on the hybrid

formulation for the quasi-static fracture of elastic solids and implemented

in the commercial FEM software ABAQUS with a user-element

subroutine (UEL).

Visualizing PCA

In order to better understand the role PCA plays in effectively

capturing the information contained in stress-strain curves, the principal

component representation of stress-strain curves is plotted in 3

dimensions. Specifically, we take the first three principal components,

which have a cumulative explained variance ~85%, and plot stress-strain

curves in that basis and provide several different angles from which to

6

view the 3D plot. Each point represents a stress-strain curve in the PCA

latent space and is colored based on the associated modulus value. it

seems that the PCA is able to spread out the curves in the latent space

based on modulus values, which suggests that this is a useful latent space

for CNN to make predictions in.

CNN model design and performance

Our CNN was a fully convolutional neural the only

dense layer was the output layer. All convolution layers used 16 filters

with a stride of 1, with a LeakyReLU activation followed by

BatchNormalization. The first 3 Conv blocks did not have 2D

MaxPooling, followed by 9 conv blocks which did have a 2D

MaxPooling layer, placed after the BatchNormalization layer. A

GlobalAveragePooling was used to reduce the dimensionality of the

output tensor from the sequential convolution blocks and the final output

layer was a Dense layer with 15 nodes, where each node corresponded to

a principal component. In total, our model had 26,319 trainable weights.

Our architecture was motivated by the recent development and

convergence onto fully-convolutional architectures for traditional

computer vision applications, where convolutions are empirically

observed to be more efficient and stable for learning as opposed to dense

layers. In addition, in our previous work, we had shown that CNN's were

7

a capable architecture for learning to predict mechanical properties of 2D

composites [30]. The convolution operation is an intuitively good fit for

predicting crack propagation because it is a local operation, allowing it to

implicitly featurize and learn the local spatial effects of crack

propagation.

After applying PCA transformation to reduce the dimensionality of

the target variable, CNN is used to predict the PCA representation of the

stress-strain curve of a given binary composite design. After training the

CNN on a training set, its ability to generalize to composite designs it has

not seen is evaluated by comparing its predictions on an unseen test set.

However, a natural question that emerges is how to evaluate a model's

performance at predicting stress-strain curves in a real-world engineering

context. While simple scaler metrics such as mean squared error (MSE)

and mean absolute error (MAE) generalize easily to vector targets, it is

not clear how to interpret these aggregate summaries of performance. It is

difficult to use such metrics to ask questions such as

enough to use in the real world”　and “On average, how poorly will a

given prediction be incorrect relative to some given specificationAlthough being able to predict stress-strain curves is an important

application of FEM and a highly desirable property for any machine

learning model to learn, it does not easily lend itself to interpretation.

Specifically, there is no simple quantitative way to define whether two

8

“Is this mode”.

stress-strain curves are “close” or “similar-world units. ” with realGiven that stress-strain curves are oftentimes intermediary

representations of a composite property that are used to derive more

meaningful descriptors such as modulus, strength, and toughness, we

decided to evaluate the model in an analogous fashion. The CNN

prediction in the PCA latent space representation is transformed back to a

stress-strain curve using PCA, and used to derive the predicted modulus,

strength, and toughness of the composite. The predicted material

descriptors are then compared with the actual material descriptors. In this

way, MSE and MAE now have clearly interpretable units and meanings.

The average performance of the model with respect to the error between

the actual and predicted material descriptor values derived from

stress-strain curves are presented in Table. The MAE for material

descriptors provides an easily interpretable metric of model performance

and can easily be used in any design specification to provide confidence

estimates of a model prediction. When comparing the mean absolute error

(MAE) to the range of values taken on by the distribution of material

descriptors, we can see that the MAE is relatively small compared to the

range. The MAE compared to the range is <10% for all material

descriptors. Relatively tight confidence intervals on the error indicate that

this model architecture is stable, the model performance is not heavily

dependent on initialization, and that our results are robust to different

9

train-test splits of the data.

Future work

Future work includes combining empirical models with optimization

algorithms, such as gradient-based methods, to identify composite

designs that yield complementary mechanical properties. The ability of a

trained empirical model to make high-throughput predictions over

designs it has never seen before allows for large parameter space

optimization that would be computationally infeasible for FEM. In

addition, we plan to explore different visualizations of empirical models

in an effort to “open up the black-box” of such models. Applying machine

learning to finite-element methods is a rapidly growing field with the

potential to discover novel next-generation materials tailored for a variety

of applications. We also note that the proposed method can be readily

applied to predict other physical properties represented in a similar

vectorized format, such as electron/phonon density of states, and

sound/light absorption spectrum.

Conclusion

In conclusion, we applied PCA and CNN to rapidly and accurately

predict the stress-strain curves of composites beyond the elastic limit. In

doing so, several novel methodological approaches were developed,

including using the derived material descriptors from the stress-strain

curves as interpretable metrics for model performance and dimensionality

10

reduction techniques to stress-strain curves. This method has the potential

to enable composite design with respect to mechanical response beyond

the elastic limit, which was previously computationally infeasible, and

can generalize easily to related problems outside of microstructural

design for enhancing mechanical properties.

中文基于卷积神经网络的复合材料微结构应力查尔斯，吉姆，瑞恩，格瑞斯摘要应力-应变曲线是材料机械性能的重要代表，从中可以定义重要的性能，例如弹性模量，强度和韧性。但是，从数值方法（例如有限元方法（FEM））生成应力-应变曲线的计算量很大，尤其是在考虑材料的整个失效路径时。结果，难以对具有较大设计空间的材料进行高通量计算设计，尤其是在考虑超出弹性极限的机械响应时。在这项工-应变曲线预测作中，主成分分析（PCA）和卷积神经网络（CNN）的组合被用于预测在整个失效路径上评估的二元复合材料的整体应力是由于经验模型的推断速度显着加快的缘故。我们展示了-应变行为，这PCA通过可视化PCA的本征基础将应力-应变曲线转换为有效的潜在空间。尽管只有可能的微观结构配置的10-27％的数据集，但在基于派生的材料描述符（例如模量，强度）测量模型性能时，预测的平均绝对误差小于数据集中值范围的10％和韧性。我们的研究表明使用机器学习11

来加速材料设计，表征和优化的潜力。关键词：机器学习，卷积神经网络，力学性能，微观结构，计算力学引言理解材料的结构与性能之间的关系是材料科学中的一个重要问题，在设计下一代材料方面有重要的应用。一个主要的动机示例是设计用于承重应用的复合材料微结构，因为复合材料可提供有利的高比强度和比韧性。增材制造的最新进展促进了复杂复合结构的制造，果，通过3D打印方法制造并测试了各种复杂设计。尽管更先进的制造技术为先进的材料和新颖的功能性开辟了前所未有的机遇，定具有所需性能的微结构却是一个困难的优化问题。确定最佳组合设计的一种方法是构建分析理论。对于常规的颗粒但要确结/纤维增强复合材料，已开发出多种均质化理论来预测复合材料的机械性能随增强材料的体积分数，纵横比和取向分布的变化。由于许多通过自组装过程合成的天然复合材料具有相对周期性和规则的结构，因此，如果了解代表性单位晶格的载荷传递机理和自相似分层结构的作用，则可以预测其机械性能。但是，在存在缺陷的情况下，分析理论的应用范围仅限于定量预测超出弹性极限的复合材料性能，因为此类理论依赖于代表体积元素（RVE）的概念，即材料性能的统计表示，而强度和强度失败取决于整个样本域中最弱的缺陷。（FEM）的数值建模可以补充用于预测非弹性属性基于有限元方法（例如强度和韧性-应模量，以下简称韧性）的分析方法，这些方法只能从完整的应力12

变曲线获得。但是，由于需要非常细的网格来适应裂纹尖端附近的高度集中的应力场，因此能够模拟曲线裂纹的萌生和扩展的数值模式（例如裂纹相场模型）在计算上是昂贵且费时的。扩散裂纹表面附近损伤参数的变化。同时，分析模型需要大量的人力和领域专业知识，并且不能推广到类似的领域问题。为了在现实的时间内在大型设计空间中识别出高性能的复合材料，我们需要能够快速描述复杂系统的机械性能并易于推广到类似系统的模型。机器学习提供了极快的推理时间的优势，并且仅需训练数据即可学习输入和输出之间的关系，例如复合微结构及其机械性能。机器学习已被应用来加速几个不同物理系统的优化，包括石墨烯kirigami切割，微调自旋qubit参数和探针显微镜微调。这样的模型不需要大量的人工干预或知识，不需要相对于输入设计空间有效地学习关系，并且可以推广到不同的系统。在本文中，我们结合主成分分析（PCA）和卷积神经网络（CNN）来预测超出弹性极限的复合材料破坏的整个应力-应变曲线。选择应力-应变曲线作为模型的目标，因为鉴于它们的高维数，它们很难预测。另外，应力-应变曲线用于导出重要的材料描述，如模量，强度和韧性。从这个意义上讲，预测应力-应变曲线是比定标器材料描述100,000个不同的复合微符的任何组合更全面的复合材料性能描述。结构及其对应的应力-应变曲线的数据集用于训练和评估模型性能。由于应力-应变数据集的高维性，因此使用了多种降维方法，包括PCA，该方法将领域理解和传统机器学习相结合，从而简化了问题，13

而又不损失模型的一般性。我们将首先描述建模方法和用于生成数据的有限元方法（的参数。然后呈现学习到的果。CNN的实施和培训卷积神经网络经过训练可以预测应力向量的这种较低维表示。CNN的输入是代表复合设计的二进制矩阵，对应于硬块。其中0对应于软块，而1Python软件包FEM）PCA潜在空间的可视化以及模型性能结PCA是使用默认的超参数通过开源scikit-learn实现的。CNN是使用Keras与TensorFlow后端实现的。所有实验的批次大小均设置为用于在反向传播期间更新16，历时数设置为30。Adam优化器CNN权重。95：5 –由于数据集相对较大，因95：5且具有使用的火车/测试拆分比率为此我们使用比标准80:20小的比率进行验证。比率为100,000个实例的数据集，测试集大小仍然具有足够的数据点（大约几千个），以便将其结果推广。将目标PCA表示的每一列标准化为平均值为0，标准差为1，以防止训练不稳定。有限元方法数据生成FEM用于生成CNN模型的训练数据。尽管最初获得的训练数据是计算密集型的，但训练CNN模型所需的时间要少得多，并且可以对成千上万个新的，随机生成的复合物进行高吞吐量推断的时间更少。裂纹相场求解器基于用于弹性固体准静态断裂的混合公式，并在带有用户元素子例程（UEL）的商业FEM软件ABAQUS中实现。14

可视化PCA

为了更好地理解PCA在有效捕获应力-应变曲线中包含的信息中所起的作用，应力-应变曲线的主成分表示形式分为3维。具体来说，我们采用前三个主要成分，它们具有?85％的累积解释方差，并在此基础上绘制应力-应变曲线，并提供几个不同的角度来查看3D绘图。每个点代表PCA潜在空间中的应力-应变曲线，并根据关联的模量值进行着色。似乎PCA能够基于模量值在潜在空间中展开曲线，这表明这是CNN进行预测的有用潜在空间。CNN模型设计与性能我们的CNN是一个全卷积神经网络，即唯一的密集层是输出层。所有卷积层都使用16个步幅为1的滤波器，并激活LeakyReLU，然后进行BatchNormalization。前三个Conv块没有2D MaxPooling，然后是9个具有2D MaxPooling层的conv块，它们位于BatchNormalization层之后。使用GlobalAveragePooling来减少顺序卷积块的输出张量的维数，最终输出层是具有15个节点的密集层，其26,319个可中每个节点对应于一个主要成分。我们的模型总共具有训练的权重。我们的体系结构是受最近的发展和向传统计算机视觉应用的全卷积体系结构融合的推动而来的，在传统卷积体系结构中，凭经验观察到，与密集层相比，卷积对于学习而言更加有效和稳定。另外，在我们之前的工作中，我们已经证明了复合材料力学性能的强大架构CNN是一种能够学习预测2D[30]。由于卷积运算是局部运算，因此15

它在直观上很适合预测裂纹扩展，因为它可以隐式地特征化并了解裂纹传播的局部空间效应。在应用PCA变换以减少目标变量的维数之后，使用CNN预测给定二元复合设计的应力-应变曲线的PCA表示。在训练集上对进行训练后，通过在一个看不见的测试集上比较其预测，推广到尚未看到的复合设计的能力。实际工程环境中评估模型在预测应力CNN可以评估其但是，出现的自然问题是如何在-应变曲线时的性能。尽管简单的定标器度量标准（例如均方误差（MSE）和均值绝对误差（MAE））可以很容易地推广到矢量目标，但尚不清楚如何解释这些综合性能汇总。很难使用此类指标来问诸如“该模型是否足以在现实世界中使用”和“平均而言，相对于某些给定的规范，给定的预测不正确的程度如何”之类的问题。尽管能够预测应力-应变曲线是FEM的重要应用，但它并不容易并且对于任何机器学习模型来说都是非常可取的特性，解释。具体而言，没有简单的定量方法来定义两条应力应变曲线是与实际单位是“接近”还是“相似”。考虑到应力-应变曲线通常是复合材料的中间表示，可用来导出更有意义的描述子，例如模量，强度和韧性，因此，我们决定以类似的方式评估模型。使用PCA将PCA潜在空间表示中的CNN预测转换回应力-应变曲线，并用于得出复合材料的预测模量，强度和韧性。然后将预测的物料描述符与实际物料描述符进行比较。这样，MSE和MAE现在具有清晰可解释的单位和含义。表中列出了相对于从应力-应变曲线得出的实际和预测材料描述符值之间的误差，模型的平16

均性能。材料描述符的MAE提供了易于理解的模型性能指标，并且可以轻松地用于任何设计规范中，以提供模型预测的置信度估计。将平均绝对误差（MAE）与材料描述符的分布所取值的范围进行比较时，我们可以看到MAE与该范围相比相对较小。对于所有材料描述符，与该范围相比的MAE <10％。误差上相对较小的置信区间表明该模型体系结构是稳定的，模型的性能在很大程度上不依赖于初始化，并且我们的结果对于数据的不同训练测试拆分具有鲁棒性。未来的研究未来的工作包括将经验模型与优化算法（例如基于梯度的方法）相结合，以识别产生互补机械性能的复合设计。经过训练的经验模型对以前从未见过的设计做出高通量预测的能力允许进行大参数空间优化，而这对于FEM而言在计算上是不可行的。此外，我们计划探索经验模型的不同可视化方法，以“打开这种模型的黑匣子”器学习应用于有限元方法是一个快速发展的领域，用量身定制的新型下一代材料的潜力。。将机具有发现为各种应我们还注意到，提出的方法可例如以很容易地应用于预测以类似矢量化格式表示的其他物理属性，状态的电子/声子密度和声音/光吸收谱。结论总之，我们应用PCA和CNN快速准确地预测了超出弹性极限的复合材料的应力-应变曲线。通过这样做，开发了几种新颖的方法学方法，包括使用从应力-应变曲线导出的材料描述符作为模型性能和应力-应变曲线的降维技术的可解释指标。这种方法有可能使复合设17

计的机械响应超出弹性极限，而弹性极限以前在计算上是不可行的，并且可以轻松地推广到微结构设计之外的相关问题以增强机械性能。文献出处：Charles Yang, Youngsoo Kim, Seunghwa Ryu, Grace X.

Gu.[J]Materials & Design, Volume 189, April 2020

18