快活林资源网

tensorflow模型转ncnn的操作方式
脚本专栏 2024/12/26 佚名
3 1
快活林资源网 Design By www.csstdc.com

第一步把tensorflow保存的.ckpt模型转为pb模型, 并记下模型的输入输出名字.

第二步去ncnn的github上把仓库clone下来, 按照上面的要求装好依赖并make.

第三步是修改ncnn的CMakeList, 具体修改的位置有:

ncnn/CMakeList.txt 文件, 在文件开头处加入add_definitions(-std=c++11), 末尾处加上add_subdirectory(examples), 如果ncnn没有examples文件夹,就新建一个, 并加上CMakeList.txt文件.

ncnn/tools/CMakeList.txt 文件, 加入add_subdirectory(tensorflow)

原版的tools/tensorflow/tensorflow2ncnn.cpp里, 不支持tensorflow的elu, FusedBathNormalization, Conv2dBackpropback操作, 其实elu是支持的,只需要仿照relu的格式, 在.cpp文件里加上就行. FusedBatchNormalization就是ncnn/layer/里实现的batchnorm.cpp, 只是`tensorflow2ncnn里没有写上, 可以增加下面的内容：

else if (node.op() == "FusedBatchNorm")
{
 fprintf(pp, "%-16s", "BatchNorm");
}
...
else if (node.op() == "FusedBatchNorm")
{
 std::cout << "node name is FusedBatchNorm" << std::endl;
 tensorflow::TensorProto tensor;
 find_tensor_proto(weights, node, tensor);
 const tensorflow::TensorShapeProto& shape = tensor.tensor_shape();

 const tensorflow::TensorProto& gamma = weights[node.input(1)];
 const tensorflow::TensorProto& Beta = weights[node.input(2)];
 const tensorflow::TensorProto& mean = weights[node.input(3)];
 const tensorflow::TensorProto& var = weights[node.input(4)];

 int channels = gamma.tensor_shape().dim(0).size(); // data size
 int dtype = gamma.dtype();

 switch (dtype){
  case 1: 
  {

   const float * gamma_tensor = reinterpret_cast<const float *>(gamma.tensor_content().c_str());
   const float * mean_data = reinterpret_cast<const float *>(mean.tensor_content().c_str());
   const float * var_data = reinterpret_cast<const float *>(var.tensor_content().c_str());
   const float * b_data = reinterpret_cast<const float *>(Beta.tensor_content().c_str());
   for (int i=0; i< channels; ++i)
   {
    fwrite(gamma_tensor+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(mean_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(var_data+i, sizeof(float), 1, bp);
   }
   for (int i=0; i< channels; ++i)
   {
    fwrite(b_data+i, sizeof(float), 1, bp);
   }
  }
  default:
   std::cerr << "Type is not supported." << std::endl;

 }
 fprintf(pp, " 0=%d", channels);

 tensorflow::AttrValue value_epsilon;
 if (find_attr_value(node, "epsilon", value_epsilon)){
  float epsilon = value_epsilon.f();
  fprintf(pp, " 1=%f", epsilon);
 }
}

同理, Conv2dBackpropback其实就是ncnn里的反卷积操作, 只不过ncnn实现反卷积的操作和tensorflow内部实现反卷积的操作过程不一样, 但结果是一致的, 需要仿照普通卷积的写法加上去.

ncnn同样支持空洞卷积, 但无法识别tensorflow的空洞卷积, 具体原理可以看tensorflow空洞卷积的原理, tensorflow是改变featuremap做空洞卷积, 而ncnn是改变kernel做空洞卷积, 结果都一样. 需要对.proto文件修改即可完成空洞卷积.

总之ncnn对tensorflow的支持很不友好, 有的层还需要自己手动去实现, 还是很麻烦.

补充知识：pytorch模型转mxnet

介绍

gluon把mxnet再进行封装，封装的风格非常接近pytorch

使用gluon的好处是非常容易把pytorch模型向mxnet转化

唯一的问题是gluon封装还不成熟，封装好的layer不多，很多常用的layer 如concat,upsampling等layer都没有

这里关注如何把pytorch 模型快速转换成 mxnet基于symbol 和 exector设计的网络

pytorch转mxnet module

关键点:

mxnet 设计网络时symbol 名称要和pytorch初始化中各网络层名称对应

torch.load()读入pytorch模型checkpoint 字典，取当中的'state_dict'元素，也是一个字典

pytorch state_dict 字典中key是网络层参数的名称，val是参数ndarray

pytorch 的参数名称的组织形式和mxnet一样，但是连接符号不同，pytorch是'.'，而mxnet是'_'比如：

pytorch '0.conv1.0.weight'
mxnet '0_conv1_0_weight'

pytorch 的参数array 和mxnet 的参数array 完全一样，只要名称对上，直接赋值即可初始化mxnet模型

需要做的有以下几点：

设计和pytorch网络对应的mxnet网络

加载pytorch checkpoint

调整pytorch checkpoint state_dict 的key名称和mxnet命名格式一致

FlowNet2S PytorchToMxnet

pytorch flownet2S 的checkpoint 可以在github上搜到

import mxnet as mx
from symbol_util import *
import pickle
 
def get_loss(data, label, loss_scale, name, get_input=False, is_sparse = False, type='stereo'):
 
 if type == 'stereo':
  data = mx.sym.Activation(data=data, act_type='relu',name=name+'relu')
 # loss
 if is_sparse:
  loss =mx.symbol.Custom(data=data, label=label, name=name, loss_scale= loss_scale, is_l1=True,
   op_type='SparseRegressionLoss')
 else:
  loss = mx.sym.MAERegressionOutput(data=data, label=label, name=name, grad_scale=loss_scale)
 return (loss,data) if get_input else loss
 
def flownet_s(loss_scale, is_sparse=False, name=''):
 img1 = mx.symbol.Variable('img1')
 img2 = mx.symbol.Variable('img2')
 data = mx.symbol.concat(img1,img2,dim=1)
 labels = {'loss{}'.format(i): mx.sym.Variable('loss{}_label'.format(i)) for i in range(0, 7)}
 # print('labels: ',labels)
 prediction = {}# a dict for loss collection
 loss = []#a list
 
 #normalize
 data = (data-125)/255
 
 # extract featrue
 conv1 = mx.sym.Convolution(data, pad=(3, 3), kernel=(7, 7), stride=(2, 2), num_filter=64, name=name + 'conv1_0')
 conv1 = mx.sym.LeakyReLU(data=conv1, act_type='leaky', slope=0.1)
 
 conv2 = mx.sym.Convolution(conv1, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=128, name=name + 'conv2_0')
 conv2 = mx.sym.LeakyReLU(data=conv2, act_type='leaky', slope=0.1)
 
 conv3a = mx.sym.Convolution(conv2, pad=(2, 2), kernel=(5, 5), stride=(2, 2), num_filter=256, name=name + 'conv3_0')
 conv3a = mx.sym.LeakyReLU(data=conv3a, act_type='leaky', slope=0.1)
 
 conv3b = mx.sym.Convolution(conv3a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=256, name=name + 'conv3_1_0')
 conv3b = mx.sym.LeakyReLU(data=conv3b, act_type='leaky', slope=0.1)
 
 conv4a = mx.sym.Convolution(conv3b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv4_0')
 conv4a = mx.sym.LeakyReLU(data=conv4a, act_type='leaky', slope=0.1)
 
 conv4b = mx.sym.Convolution(conv4a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv4_1_0')
 conv4b = mx.sym.LeakyReLU(data=conv4b, act_type='leaky', slope=0.1)
 
 conv5a = mx.sym.Convolution(conv4b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=512, name=name + 'conv5_0')
 conv5a = mx.sym.LeakyReLU(data=conv5a, act_type='leaky', slope=0.1)
 
 conv5b = mx.sym.Convolution(conv5a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=512, name=name + 'conv5_1_0')
 conv5b = mx.sym.LeakyReLU(data=conv5b, act_type='leaky', slope=0.1)
 
 conv6a = mx.sym.Convolution(conv5b, pad=(1, 1), kernel=(3, 3), stride=(2, 2), num_filter=1024, name=name + 'conv6_0')
 conv6a = mx.sym.LeakyReLU(data=conv6a, act_type='leaky', slope=0.1)
 
 conv6b = mx.sym.Convolution(conv6a, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=1024,
        name=name + 'conv6_1_0')
 conv6b = mx.sym.LeakyReLU(data=conv6b, act_type='leaky', slope=0.1, )
 
 #predict flow
 pr6 = mx.sym.Convolution(conv6b, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow6')
 prediction['loss6'] = pr6
 
 upsample_pr6to5 = mx.sym.Deconvolution(pr6, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow6_to_5', no_bias=True)
 upconv5 = mx.sym.Deconvolution(conv6b, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=512,
         name=name + 'deconv5_0', no_bias=False)
 upconv5 = mx.sym.LeakyReLU(data=upconv5, act_type='leaky', slope=0.1)
 iconv5 = mx.sym.Concat(conv5b, upconv5, upsample_pr6to5, dim=1)
 
 
 pr5 = mx.sym.Convolution(iconv5, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow5')
 prediction['loss5'] = pr5
 
 upconv4 = mx.sym.Deconvolution(iconv5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=256,
         name=name + 'deconv4_0', no_bias=False)
 upconv4 = mx.sym.LeakyReLU(data=upconv4, act_type='leaky', slope=0.1)
 
 upsample_pr5to4 = mx.sym.Deconvolution(pr5, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow5_to_4', no_bias=True)
 
 iconv4 = mx.sym.Concat(conv4b, upconv4, upsample_pr5to4)
 
 pr4 = mx.sym.Convolution(iconv4, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow4')
 prediction['loss4'] = pr4
 
 upconv3 = mx.sym.Deconvolution(iconv4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=128,
         name=name + 'deconv3_0', no_bias=False)
 upconv3 = mx.sym.LeakyReLU(data=upconv3, act_type='leaky', slope=0.1)
 
 upsample_pr4to3 = mx.sym.Deconvolution(pr4, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name= name + 'upsampled_flow4_to_3', no_bias=True)
 iconv3 = mx.sym.Concat(conv3b, upconv3, upsample_pr4to3)
 
 pr3 = mx.sym.Convolution(iconv3, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow3')
 prediction['loss3'] = pr3
 
 upconv2 = mx.sym.Deconvolution(iconv3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=64,
         name=name + 'deconv2_0', no_bias=False)
 upconv2 = mx.sym.LeakyReLU(data=upconv2, act_type='leaky', slope=0.1)
 
 upsample_pr3to2 = mx.sym.Deconvolution(pr3, pad=(1, 1), kernel=(4, 4), stride=(2, 2), num_filter=2,
           name=name + 'upsampled_flow3_to_2', no_bias=True)
 iconv2 = mx.sym.Concat(conv2, upconv2, upsample_pr3to2)
 
 pr2 = mx.sym.Convolution(iconv2, pad=(1, 1), kernel=(3, 3), stride=(1, 1), num_filter=2,
        name=name + 'predict_flow2')
 prediction['loss2'] = pr2
 flow = mx.sym.UpSampling(arg0=pr2,scale=4,num_filter=2,num_args = 1,sample_type='nearest', name='upsample_flow2_to_1')
 # ignore the loss functions with loss scale of zero
 keys = loss_scale.keys()
 # keys.sort()
 #obtain the symbol of the losses
 for key in keys:
  # loss.append(get_loss(prediction[key] * 20, labels[key], loss_scale[key], name=key + name,get_input=False, is_sparse=is_sparse, type='flow'))
  loss.append(mx.sym.MAERegressionOutput(data=prediction[key] * 20, label=labels[key], name=key + name, grad_scale=loss_scale[key]))
 # print('loss: ',loss)
 #group 暂时不知道为嘛要group
 loss_group =mx.sym.Group(loss)
 # print('net: ',loss_group)
 return loss_group,flow
 
import gluonbook as gb
import torch
from utils.frame_utils import *
import numpy as np
if __name__ == '__main__':
 checkpoint = torch.load("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/flownet2_pytorch/FlowNet2-S_checkpoint.pth.tar")
 # # checkpoint是一个字典
 print(isinstance(checkpoint['state_dict'], dict))
 # # 打印checkpoint字典中的key名
 print('keys of checkpoint:')
 for i in checkpoint:
  print(i)
 print('')
 # # pytorch 模型参数保存在一个key名为'state_dict'的元素中
 state_dict = checkpoint['state_dict']
 # # state_dict也是一个字典
 print('keys of state_dict:')
 for i in state_dict:
  print(i)
  # print(state_dict[i].size())
 print('')
 # print(state_dict)
 #字典的value是torch.tensor
 print(torch.is_tensor(state_dict['conv1.0.weight']))
 #查看某个value的size
 print(state_dict['conv1.0.weight'].size())
 
 #flownet-mxnet init
 loss_scale={'loss2': 1.00,
    'loss3': 1.00,
    'loss4': 1.00,
    'loss5': 1.00,
    'loss6': 1.00}
 loss,flow = flownet_s(loss_scale=loss_scale,is_sparse=False)
 print('loss information: ')
 print('loss:',loss)
 print('type:',type(loss))
 print('list_arguments:',loss.list_arguments())
 print('list_outputs:',loss.list_outputs())
 print('list_inputs:',loss.list_inputs())
 print('')
 
 print('flow information: ')
 print('flow:',flow)
 print('type:',type(flow))
 print('list_arguments:',flow.list_arguments())
 print('list_outputs:',flow.list_outputs())
 print('list_inputs:',flow.list_inputs())
 print('')
 name_mxnet = symbol.list_arguments()
 print(type(name_mxnet))
 for key in name_mxnet:
  print(key)
 
 name_mxnet.sort()
 for key in name_mxnet:
  print(key)
 print(name_mxnet)
 
 shapes = (1, 3, 384, 512)
 ctx = gb.try_gpu()
 # exe = symbol.simple_bind(ctx=ctx, img1=shapes,img2=shapes)
 exe = flow.simple_bind(ctx=ctx, img1=shapes, img2=shapes)
 print('exe type: ',type(exe))
 print('exe: ',exe)
 #module
 # mod = mx.mod.Module(flow)
 # print('mod type: ', type(exe))
 # print('mod: ', exe)
 
 pim1 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img0.ppm")
 pim2 = read_gen("C:/Users/junjie.huang/PycharmProjects/flownet2_mxnet/data/0000007-img1.ppm")
 print(pim1.shape)
 
 '''使用pytorch 的state_dict 初始化 mxnet 模型参数'''
 for key in state_dict:
  # print(type(key))
  k_split = key.split('.')
  key_mx = '_'.join(k_split)
  # print(key,key_mx)
  try:
   exe.arg_dict[key_mx][:]=state_dict[key].data
  except:
   print(key,exe.arg_dict[key_mx].shape,state_dict[key].data.shape)
 
 exe.arg_dict['img1'][:] = pim1[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 exe.arg_dict['img2'][:] = pim2[np.newaxis, :, :, :].transpose(0, 3, 1, 2).data
 
 result = exe.forward()
 print('result: ',type(result))
 # for tmp in result:
 #  print(type(tmp))
 #  print(tmp.shape)
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 outputs = exe.outputs
 print('output type: ',type(outputs))
 # for tmp in outputs:
 #  print(type(tmp))
 #  print(tmp.shape)
 
 #来自pytroch flownet2
 from visualize import flow2color
 # color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1,2,0))
 flow_color = flow2color(exe.outputs[0].asnumpy()[0].transpose(1, 2, 0))
 print('color type:',type(flow_color))
 import matplotlib.pyplot as plt
 #来自pytorch
 from torchvision.transforms import ToPILImage
 TF = ToPILImage()
 images = TF(flow_color)
 images.show()
 # plt.imshow(color)

以上这篇tensorflow模型转ncnn的操作方式就是小编分享给大家的全部内容了，希望能给大家一个参考，也希望大家多多支持。

快活林资源网 Design By www.csstdc.com
广告合作：本站广告合作请联系QQ：858582 申请时备注：广告合作（否则不回）
免责声明：本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除！
快活林资源网 Design By www.csstdc.com
www.csstdc.com 快活林资源网
120,135影音资源
344,641技术资源
22,817软件资源
435,032站长资源

P70系列延期，华为新旗舰将在下月发布

3月20日消息，近期博主@数码闲聊站 透露，原定三月份发布的华为新旗舰P70系列延期发布，预计4月份上市。

而博主@定焦数码 爆料，华为的P70系列在定位上已经超过了Mate60，成为了重要的旗舰系列之一。它肩负着重返影像领域顶尖的使命。那么这次P70会带来哪些令人惊艳的创新呢？

根据目前爆料的消息来看，华为P70系列将推出三个版本，其中P70和P70 Pro采用了三角形的摄像头模组设计，而P70 Art则采用了与上一代P60 Art相似的不规则形状设计。这样的外观是否好看见仁见智，但辨识度绝对拉满。

更新日志

友情链接

杰晶网络 DDR爱好者之家 桃源资源网 杰网资源 富贵资源网 南强小屋 铁雪资源网 幽灵资源网 万梅资源网 狼山资源网 白云岛资源网 昆仑资源网 相思资源网 明霞山资源网 内蒙古资源网 黑松山资源网 茶园资源网 饿虎岗资源网 大旗谷资源网 常春岛资源网 岱庙资源网 兴国资源网 快活林资源网 蝙蝠岛资源网 帝王谷资源网 白云城资源网 伏龙阁资源网 清风细雨楼 天枫庄资源网 圆月山庄资源网 无争山庄资源网 神水资源网 移花宫资源网 神剑山庄资源网 无为清净楼资源网 金钱帮资源网 丐帮资源网 华山资源网 极乐门资源网 小李飞刀资源网 凤求凰客栈 风云阁资源网 金狮镖局 鸳鸯亭资源网 千金楼资源网 更多链接
快活林资源网 Design By www.csstdc.com