基于Hadoop MapReduce的电商购物数据分析平台实战

本文将详细介绍一个完整的电商购物数据分析平台项目，从数据采集、MapReduce处理、Spring Boot后端服务到Vue.js前端可视化展示的全流程实现。

一、项目背景与概述

随着电商业务的快速发展，企业积累了海量的购物数据。如何从这些数据中提取有价值的信息，为业务决策提供支持，成为电商企业面临的重要挑战。本项目旨在利用大数据技术对购物数据进行多维度分析，通过可视化方式展示分析结果，帮助企业更好地理解客户需求和市场趋势。

1.1 项目技术栈

层次	技术	版本	用途
数据处理层	Hadoop MapReduce	3.3.4	大数据处理
数据处理层	Java	1.8+	开发语言
服务层	Spring Boot	2.7.18	后端框架
展示层	Vue.js	3.5.26	前端框架
展示层	Element Plus	2.13.1	UI组件库
展示层	ECharts	5.5.1	数据可视化

1.2 项目架构

┌─────────────────────────────────────────────────┐
│                 展示层 (前端)                     │
│  Vue.js 3 + Element Plus + ECharts              │
└─────────────────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────┐
│                 服务层 (后端)                     │
│  Spring Boot + RESTful API                       │
└─────────────────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────┐
│               数据处理层 (MapReduce)             │
│  Hadoop MapReduce + Java                         │
└─────────────────────────────────────────────────┘
                        │
                        ▼
┌─────────────────────────────────────────────────┐
│                 存储层                          │
│  CSV数据文件 + 分析结果文件                       │
└─────────────────────────────────────────────────┘

二、数据集介绍

本项目使用电商购物数据集，数据格式为CSV，包含以下字段：

字段名	说明	示例
invoice_no	发票编号	I138884
customer_id	客户ID	C241288
gender	性别	Female/Male
age	年龄	28
category	商品类别	Clothing/Shoes/Books等
quantity	购买数量	5
price	单价	1500.4
payment_method	支付方式	Alipay/WeChat Pay/Card
invoice_date	发票日期	5/8/2022

数据示例：

invoice_no,customer_id,gender,age,category,quantity,price,payment_method,invoice_date
I138884,C241288,Female,28,Clothing,5,1500.4,Alipay,5/8/2022
I317333,C111565,Male,21,Shoes,3,1800.51,WeChat Pay,12/12/2021
I127801,C266599,Male,20,Clothing,1,300.08,Card,9/11/2021

三、MapReduce数据处理模块

3.1 核心组件设计

MapReduce模块负责对原始CSV数据进行多维度分析，生成各类分析结果文件。

3.1.1 数据解析工具类

public class ShoppingParser {
    private String invoiceNo;
    private String customerId;
    private String gender;
    private int age;
    private String category;
    private int quantity;
    private double price;
    private String paymentMethod;
    private String invoiceDate;
    private double totalAmount;

    public ShoppingParser(String csvLine) {
        parseCSVLine(csvLine);
        this.totalAmount = this.quantity * this.price;
    }

    private void parseCSVLine(String line) {
        String[] fields = line.split(",");
        this.invoiceNo = fields[0];
        this.customerId = fields[1];
        this.gender = fields[2];
        this.age = Integer.parseInt(fields[3]);
        this.category = fields[4];
        this.quantity = Integer.parseInt(fields[5]);
        this.price = Double.parseDouble(fields[6]);
        this.paymentMethod = fields[7];
        this.invoiceDate = fields[8];
    }

    public double getTotalAmount() {
        return totalAmount;
    }

    // getter方法...
}

3.1.2 自定义Hadoop数据类型

public class ShoppingSalesWritable implements Writable {
    private DoubleWritable totalSales = new DoubleWritable();
    private DoubleWritable averagePrice = new DoubleWritable();
    private DoubleWritable totalQuantity = new DoubleWritable();

    public ShoppingSalesWritable() {}

    public ShoppingSalesWritable(double totalSales, double averagePrice, double totalQuantity) {
        this.totalSales.set(totalSales);
        this.averagePrice.set(averagePrice);
        this.totalQuantity.set(totalQuantity);
    }

    @Override
    public void write(DataOutput out) throws IOException {
        totalSales.write(out);
        averagePrice.write(out);
        totalQuantity.write(out);
    }

    @Override
    public void readFields(DataInput in) throws IOException {
        totalSales.readFields(in);
        averagePrice.readFields(in);
        totalQuantity.readFields(in);
    }

    public void add(ShoppingSalesWritable other) {
        double newTotalSales = this.totalSales.get() + other.totalSales.get();
        double newTotalQuantity = this.totalQuantity.get() + other.totalQuantity.get();
        double newAvgPrice = newTotalQuantity > 0 ? newTotalSales / newTotalQuantity : 0;
        
        this.totalSales.set(newTotalSales);
        this.averagePrice.set(newAvgPrice);
        this.totalQuantity.set(newTotalQuantity);
    }
}

3.2 Mapper实现

3.2.1 商品类别总销售额Mapper

public class CategoryTotalSalesMapper extends Mapper<LongWritable, Text, Text, DoubleWritable> {
    private Text categoryKey = new Text();
    private DoubleWritable salesValue = new DoubleWritable();

    @Override
    protected void map(LongWritable key, Text value, Context context) 
            throws IOException, InterruptedException {
        if (key.get() == 0) {
            return;
        }

        ShoppingParser shopping = new ShoppingParser(value.toString());
        String category = shopping.getCategory();
        double totalSales = shopping.getTotalAmount();

        categoryKey.set(category);
        salesValue.set(totalSales);
        context.write(categoryKey, salesValue);
    }
}

3.2.2 年龄段商品类别销售数量Mapper

public class AgeGroupCategorySalesMapper extends Mapper<LongWritable, Text, Text, IntWritable> {
    private Text key = new Text();
    private IntWritable value = new IntWritable();

    @Override
    protected void map(LongWritable key, Text value, Context context) 
            throws IOException, InterruptedException {
        if (key.get() == 0) {
            return;
        }

        ShoppingParser shopping = new ShoppingParser(value.toString());
        String ageGroup = getAgeGroup(shopping.getAge());
        String category = shopping.getCategory();
        int quantity = shopping.getQuantity();

        this.key.set(ageGroup + "\t" + category);
        this.value.set(quantity);
        context.write(this.key, this.value);
    }

    private String getAgeGroup(int age) {
        if (age <= 25) return "18-25岁";
        else if (age <= 35) return "26-35岁";
        else if (age <= 45) return "36-45岁";
        else if (age <= 55) return "46-55岁";
        else return "56岁以上";
    }
}

3.3 Reducer实现

3.3.1 商品类别总销售额Reducer

public class CategoryTotalSalesReducer extends Reducer<Text, DoubleWritable, Text, Text> {
    private Text result = new Text();
    private DecimalFormat df = new DecimalFormat("0.00");

    @Override
    protected void reduce(Text key, Iterable<DoubleWritable> values, Context context) 
            throws IOException, InterruptedException {
        double sum = 0;
        for (DoubleWritable value : values) {
            sum += value.get();
        }
        result.set(df.format(sum));
        context.write(key, result);
    }
}

3.3.2 商品类别销售详情Reducer

public class CategorySalesReducer extends Reducer<Text, ShoppingSalesWritable, Text, Text> {
    private Text result = new Text();
    private DecimalFormat df = new DecimalFormat("0.00");

    @Override
    protected void reduce(Text key, Iterable<ShoppingSalesWritable> values, Context context) 
            throws IOException, InterruptedException {
        ShoppingSalesWritable sum = new ShoppingSalesWritable();
        
        for (ShoppingSalesWritable value : values) {
            sum.add(value);
        }

        String output = String.format("%.2f\t%.2f\t%.0f", 
            sum.getTotalSales().get(), 
            sum.getAveragePrice().get(), 
            sum.getTotalQuantity().get());
        
        result.set(output);
        context.write(key, result);
    }
}

3.4 驱动程序实现

public class ShoppingAnalysisDriver {
    public static void main(String[] args) throws Exception {
        Configuration conf = new Configuration();
        conf.set("fs.defaultFS", "file:///");
        conf.set("mapreduce.framework.name", "local");

        String inputPath = "data/customer_shopping_data.csv";
        String categoryOutputPath = "output/category_sales";
        String paymentMethodOutputPath = "output/payment_method_sales";
        String categoryDetailOutputPath = "output/category_detail_sales";
        String genderDetailOutputPath = "output/gender_detail_sales";
        String ageGroupCategoryOutputPath = "output/age_group_category_sales";
        String ageGroupCategoryTotalSalesOutputPath = "output/age_group_category_total_sales";
        String genderCategorySalesCountOutputPath = "output/gender_category_sales_count";
        String genderCategoryTotalSalesOutputPath = "output/gender_category_total_sales";

        boolean categorySuccess = runCategoryTotalSalesJob(conf, inputPath, categoryOutputPath);
        boolean paymentMethodSuccess = runPaymentMethodTotalSalesJob(conf, inputPath, paymentMethodOutputPath);
        boolean categoryDetailSuccess = runCategorySalesJob(conf, inputPath, categoryDetailOutputPath);
        boolean genderDetailSuccess = runGenderSalesJob(conf, inputPath, genderDetailOutputPath);
        boolean ageGroupCategorySuccess = runAgeGroupCategorySalesJob(conf, inputPath, ageGroupCategoryOutputPath);
        boolean ageGroupCategoryTotalSalesSuccess = runAgeGroupCategoryTotalSalesJob(conf, inputPath, ageGroupCategoryTotalSalesOutputPath);
        boolean genderCategorySalesCountSuccess = runGenderCategorySalesCountJob(conf, inputPath, genderCategorySalesCountOutputPath);
        boolean genderCategoryTotalSalesSuccess = runGenderCategoryTotalSalesJob(conf, inputPath, genderCategoryTotalSalesOutputPath);

        if (categorySuccess && paymentMethodSuccess && categoryDetailSuccess && genderDetailSuccess 
                && ageGroupCategorySuccess && ageGroupCategoryTotalSalesSuccess 
                && genderCategorySalesCountSuccess && genderCategoryTotalSalesSuccess) {
            System.out.println("所有MapReduce作业执行成功！");
        } else {
            System.out.println("部分或所有MapReduce作业执行失败！");
            System.exit(1);
        }
    }

    private static boolean runCategoryTotalSalesJob(Configuration conf, String inputPath, String outputPath) 
            throws Exception {
        Job job = Job.getInstance(conf, "Category Total Sales Analysis");
        job.setJarByClass(ShoppingAnalysisDriver.class);
        job.setMapperClass(CategoryTotalSalesMapper.class);
        job.setReducerClass(CategoryTotalSalesReducer.class);
        job.setMapOutputKeyClass(Text.class);
        job.setMapOutputValueClass(DoubleWritable.class);
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
        FileInputFormat.addInputPath(job, new Path(inputPath));
        FileOutputFormat.setOutputPath(job, new Path(outputPath));
        return job.waitForCompletion(true);
    }

    // 其他作业配置方法...
}

3.5 分析任务汇总

分析任务	Mapper类	Reducer类	输出格式
商品类别总销售额	CategoryTotalSalesMapper	CategoryTotalSalesReducer	类别\t总销售额
支付方式总销售额	PaymentMethodTotalSalesMapper	PaymentMethodTotalSalesReducer	支付方式\t总销售额
商品类别销售详情	CategorySalesMapper	CategorySalesReducer	类别\t总销售额\t平均价格\t总数量
性别销售详情	GenderSalesMapper	GenderSalesReducer	性别\t总销售额\t平均价格\t总数量
年龄段商品类别销售数量	AgeGroupCategorySalesMapper	AgeGroupCategorySalesReducer	年龄段\t类别\t数量
年龄段商品类别消费总额	AgeGroupCategoryTotalSalesMapper	AgeGroupCategoryTotalSalesReducer	年龄段\t类别\t总销售额
性别商品类别销售数量	GenderCategorySalesCountMapper	GenderCategorySalesCountReducer	性别\t类别\t数量
性别商品类别消费总额	GenderCategoryTotalSalesMapper	GenderCategoryTotalSalesReducer	性别\t类别\t总销售额

四、Spring Boot API服务模块

4.1 项目配置

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.7.18</version>
</parent>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
</dependencies>

4.2 数据模型设计

@Service
public class AnalysisResultService {
    private static final String OUTPUT_BASE_PATH = "F:/AAproject/customerShoppingAnalysis/output/";

    public static class ResultItem {
        private String key;
        private Object value;
        
        public ResultItem(String key, Object value) {
            this.key = key;
            this.value = value;
        }
        
        // getter/setter...
    }

    public static class TwoFieldResultItem {
        private String field1;
        private String field2;
        private Object value;
        
        public TwoFieldResultItem(String field1, String field2, Object value) {
            this.field1 = field1;
            this.field2 = field2;
            this.value = value;
        }
        
        // getter/setter...
    }

    public static class SalesDetailResultItem {
        private String key;
        private Double totalSales;
        private Double avgPrice;
        private Integer totalQuantity;
        
        public SalesDetailResultItem(String key, Double totalSales, Double avgPrice, Integer totalQuantity) {
            this.key = key;
            this.totalSales = totalSales;
            this.avgPrice = avgPrice;
            this.totalQuantity = totalQuantity;
        }
        
        // getter/setter...
    }
}

4.3 业务逻辑实现

@Service
public class AnalysisResultService {
    
    public List<ResultItem> getCategoryTotalSalesResult() {
        return readResultFileToList("category_sales/part-r-00000");
    }

    public List<SalesDetailResultItem> getCategorySalesDetailResult() {
        return readMixedDelimiterSalesDetailFile("category_detail_sales/part-r-00000");
    }

    public List<TwoFieldResultItem> getAgeGroupCategorySalesResult() {
        return readTwoFieldResultFile("age_group_category_sales/part-r-00000");
    }

    private List<ResultItem> readResultFileToList(String filePath) {
        List<ResultItem> resultList = new ArrayList<>();
        try (BufferedReader reader = new BufferedReader(new FileReader(OUTPUT_BASE_PATH + filePath))) {
            String line;
            while ((line = reader.readLine()) != null) {
                String[] parts = line.split("\\s+");
                if (parts.length >= 2) {
                    String key = parts[0];
                    Object value = parts[1];
                    resultList.add(new ResultItem(key, value));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return resultList;
    }

    private List<SalesDetailResultItem> readMixedDelimiterSalesDetailFile(String filePath) {
        List<SalesDetailResultItem> resultList = new ArrayList<>();
        try (BufferedReader reader = new BufferedReader(new FileReader(OUTPUT_BASE_PATH + filePath))) {
            String line;
            while ((line = reader.readLine()) != null) {
                String[] parts = line.split("\\s+");
                if (parts.length >= 4) {
                    String key = parts[0];
                    Double totalSales = Double.parseDouble(parts[1]);
                    Double avgPrice = Double.parseDouble(parts[2]);
                    Integer totalQuantity = Integer.parseInt(parts[3]);
                    resultList.add(new SalesDetailResultItem(key, totalSales, avgPrice, totalQuantity));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return resultList;
    }

    private List<TwoFieldResultItem> readTwoFieldResultFile(String filePath) {
        List<TwoFieldResultItem> resultList = new ArrayList<>();
        try (BufferedReader reader = new BufferedReader(new FileReader(OUTPUT_BASE_PATH + filePath))) {
            String line;
            while ((line = reader.readLine()) != null) {
                String[] parts = line.split("\\s+");
                if (parts.length >= 3) {
                    String field1 = parts[0];
                    String field2 = parts[1];
                    Object value = parts[2];
                    resultList.add(new TwoFieldResultItem(field1, field2, value));
                }
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
        return resultList;
    }
}

4.4 RESTful API控制器

@RestController
@RequestMapping("/analysis")
public class AnalysisResultController {

    @Autowired
    private AnalysisResultService analysisResultService;

    @GetMapping("/category-sales-detail")
    public List<AnalysisResultService.SalesDetailResultItem> getCategorySalesDetail() {
        return analysisResultService.getCategorySalesDetailResult();
    }

    @GetMapping("/category-total-sales")
    public List<AnalysisResultService.ResultItem> getCategoryTotalSales() {
        return analysisResultService.getCategoryTotalSalesResult();
    }

    @GetMapping("/payment-method-total-sales")
    public List<AnalysisResultService.ResultItem> getPaymentMethodTotalSales() {
        return analysisResultService.getPaymentMethodTotalSalesResult();
    }

    @GetMapping("/gender-sales-detail")
    public List<AnalysisResultService.SalesDetailResultItem> getGenderSalesDetail() {
        return analysisResultService.getGenderSalesDetailResult();
    }

    @GetMapping("/age-group-category-sales")
    public List<AnalysisResultService.TwoFieldResultItem> getAgeGroupCategorySales() {
        return analysisResultService.getAgeGroupCategorySalesResult();
    }

    @GetMapping("/age-group-category-total-sales")
    public List<AnalysisResultService.TwoFieldResultItem> getAgeGroupCategoryTotalSales() {
        return analysisResultService.getAgeGroupCategoryTotalSalesResult();
    }

    @GetMapping("/gender-category-sales-count")
    public List<AnalysisResultService.TwoFieldResultItem> getGenderCategorySalesCount() {
        return analysisResultService.getGenderCategorySalesCountResult();
    }

    @GetMapping("/gender-category-total-sales")
    public List<AnalysisResultService.TwoFieldResultItem> getGenderCategoryTotalSales() {
        return analysisResultService.getGenderCategoryTotalSalesResult();
    }
}

4.5 Web配置

@Configuration
public class WebConfig implements WebMvcConfigurer {
    @Override
    public void addCorsMappings(CorsRegistry registry) {
        registry.addMapping("/**")
                .allowedOriginPatterns("*")
                .allowedMethods("GET", "POST", "PUT", "DELETE", "OPTIONS")
                .allowedHeaders("*")
                .allowCredentials(true)
                .maxAge(3600);
    }
}

五、Vue.js前端仪表板模块

5.1 项目结构

src/
├── api/                    # API请求封装
│   └── index.js            # API请求函数
├── router/                 # 路由配置
│   └── index.js            # 路由定义
├── views/                  # 页面组件
│   ├── HomeView.vue        # 首页
│   ├── CategorySalesDetailView.vue      # 商品类别销售详情
│   ├── CategoryTotalSalesView.vue        # 商品类别总销售
│   ├── PaymentMethodTotalSalesView.vue   # 支付方式总销售
│   ├── GenderSalesDetailView.vue         # 性别销售详情
│   ├── AgeGroupCategorySalesView.vue     # 年龄组类别销售数量
│   ├── AgeGroupCategoryTotalSalesView.vue# 年龄组类别总销售额
│   ├── GenderCategorySalesCountView.vue  # 性别类别销售数量
│   └── GenderCategoryTotalSalesView.vue  # 性别类别总销售额
├── App.vue                 # 根组件
└── main.js                 # 应用入口

5.2 API请求封装

import axios from 'axios';

const API_BASE_URL = 'http://localhost:8080/analysis';

export const analysisApi = {
  getCategorySalesDetail() {
    return axios.get(`${API_BASE_URL}/category-sales-detail`);
  },

  getCategoryTotalSales() {
    return axios.get(`${API_BASE_URL}/category-total-sales`);
  },

  getPaymentMethodTotalSales() {
    return axios.get(`${API_BASE_URL}/payment-method-total-sales`);
  },

  getGenderSalesDetail() {
    return axios.get(`${API_BASE_URL}/gender-sales-detail`);
  },

  getAgeGroupCategorySales() {
    return axios.get(`${API_BASE_URL}/age-group-category-sales`);
  },

  getAgeGroupCategoryTotalSales() {
    return axios.get(`${API_BASE_URL}/age-group-category-total-sales`);
  },

  getGenderCategorySalesCount() {
    return axios.get(`${API_BASE_URL}/gender-category-sales-count`);
  },

  getGenderCategoryTotalSales() {
    return axios.get(`${API_BASE_URL}/gender-category-total-sales`);
  }
};

5.3 商品类别总销售额页面

<template>
  <div class="category-total-sales-container">
    <h2 class="page-title">商品类别总销售额分析</h2>
    
    <el-card class="chart-card" shadow="hover">
      <template #header>
        <div class="card-header">
          <span>商品类别总销售额</span>
        </div>
      </template>
      <div id="categoryTotalSalesChart" ref="chartRef"></div>
    </el-card>
    
    <el-card class="table-card" shadow="hover">
      <template #header>
        <div class="card-header">
          <span>商品类别总销售额数据</span>
        </div>
      </template>
      <el-table :data="categorySalesData" style="width: 100%">
        <el-table-column prop="key" label="商品类别" width="200">
          <template #default="scope">
            <span class="category-name">{{ scope.row.key }}</span>
          </template>
        </el-table-column>
        <el-table-column prop="value" label="总销售额">
          <template #default="scope">
            <span class="sales-value">{{ formatCurrency(scope.row.value) }}</span>
          </template>
        </el-table-column>
      </el-table>
    </el-card>
  </div>
</template>

<script setup>
import { ref, onMounted, nextTick } from 'vue';
import { analysisApi } from '@/api';
import * as echarts from 'echarts';

const chartRef = ref(null);
const categorySalesData = ref([]);
let chart = null;

const formatCurrency = (value) => {
  return `¥${Number(value).toFixed(2).replace(/\B(?=(\d{3})+(?!\d))/g, ',')}`;
};

const initChart = () => {
  if (chartRef.value) {
    chart = echarts.init(chartRef.value);
    
    const option = {
      tooltip: {
        trigger: 'axis',
        axisPointer: {
          type: 'shadow'
        },
        formatter: '{b}<br/>总销售额: {c}'
      },
      xAxis: {
        type: 'category',
        data: categorySalesData.value.map(item => item.key),
        axisLabel: {
          rotate: 45
        }
      },
      yAxis: {
        type: 'value',
        axisLabel: {
          formatter: '¥{value}'
        }
      },
      series: [{
        data: categorySalesData.value.map(item => Number(item.value)),
        type: 'bar',
        itemStyle: {
          color: '#409EFF'
        }
      }]
    };
    
    chart.setOption(option);
    
    window.addEventListener('resize', () => {
      chart.resize();
    });
  }
};

const loadData = async () => {
  try {
    const response = await analysisApi.getCategoryTotalSales();
    categorySalesData.value = response.data;
    await nextTick();
    initChart();
  } catch (error) {
    console.error('加载数据失败:', error);
  }
};

onMounted(() => {
  loadData();
});
</script>

<style scoped>
.category-total-sales-container {
  max-width: 1200px;
  margin: 0 auto;
  padding: 20px;
}

.page-title {
  text-align: center;
  margin-bottom: 20px;
  color: #333;
}

.chart-card {
  margin-bottom: 20px;
  height: 500px;
}

.table-card {
  margin-bottom: 20px;
}

.card-header {
  display: flex;
  justify-content: space-between;
  align-items: center;
}

#categoryTotalSalesChart {
  width: 100%;
  height: 100%;
}

.category-name {
  font-weight: bold;
  color: #606266;
}

.sales-value {
  color: #409EFF;
  font-weight: bold;
}
</style>

5.4 年龄组类别销售数量页面（热力图）

<template>
  <div class="age-group-category-sales-container">
    <h2 class="page-title">年龄组商品类别销售数量分析</h2>
    
    <el-card class="chart-card" shadow="hover">
      <template #header>
        <div class="card-header">
          <span>年龄组商品类别销售数量热力图</span>
        </div>
      </template>
      <div id="ageGroupCategoryChart" ref="chartRef"></div>
    </el-card>
    
    <el-card class="table-card" shadow="hover">
      <template #header>
        <div class="card-header">
          <span>年龄组商品类别销售数量数据</span>
        </div>
      </template>
      <el-table :data="salesData" style="width: 100%">
        <el-table-column prop="field1" label="年龄组" width="150" />
        <el-table-column prop="field2" label="商品类别" width="200" />
        <el-table-column prop="value" label="销售数量">
          <template #default="scope">
            <span class="sales-count">{{ scope.row.value }}</span>
          </template>
        </el-table-column>
      </el-table>
    </el-card>
  </div>
</template>

<script setup>
import { ref, onMounted, nextTick } from 'vue';
import { analysisApi } from '@/api';
import * as echarts from 'echarts';

const chartRef = ref(null);
const salesData = ref([]);
let chart = null;

const initChart = () => {
  if (chartRef.value) {
    chart = echarts.init(chartRef.value);
    
    const ageGroups = [...new Set(salesData.value.map(item => item.field1))];
    const categories = [...new Set(salesData.value.map(item => item.field2))];
    
    const data = salesData.value.map(item => [
      ageGroups.indexOf(item.field1),
      categories.indexOf(item.field2),
      item.value
    ]);
    
    const option = {
      tooltip: {
        position: 'top',
        formatter: (params) => {
          return `${ageGroups[params.value[0]]} - ${categories[params.value[1]]}<br/>销售数量: ${params.value[2]}`;
        }
      },
      grid: {
        height: '70%',
        top: '10%'
      },
      xAxis: {
        type: 'category',
        data: categories,
        splitArea: {
          show: true
        }
      },
      yAxis: {
        type: 'category',
        data: ageGroups,
        splitArea: {
          show: true
        }
      },
      visualMap: {
        min: 0,
        max: Math.max(...salesData.value.map(item => item.value)),
        calculable: true,
        orient: 'horizontal',
        left: 'center',
        bottom: '0%',
        inRange: {
          color: ['#50a3ba', '#eac736', '#d94e5d']
        }
      },
      series: [{
        type: 'heatmap',
        data: data,
        label: {
          show: true
        },
        emphasis: {
          itemStyle: {
            shadowBlur: 10,
            shadowColor: 'rgba(0, 0, 0, 0.5)'
          }
        }
      }]
    };
    
    chart.setOption(option);
    
    window.addEventListener('resize', () => {
      chart.resize();
    });
  }
};

const loadData = async () => {
  try {
    const response = await analysisApi.getAgeGroupCategorySales();
    salesData.value = response.data;
    await nextTick();
    initChart();
  } catch (error) {
    console.error('加载数据失败:', error);
  }
};

onMounted(() => {
  loadData();
});
</script>

<style scoped>
.age-group-category-sales-container {
  max-width: 1200px;
  margin: 0 auto;
  padding: 20px;
}

.page-title {
  text-align: center;
  margin-bottom: 20px;
  color: #333;
}

.chart-card {
  margin-bottom: 20px;
  height: 600px;
}

.table-card {
  margin-bottom: 20px;
}

.card-header {
  display: flex;
  justify-content: space-between;
  align-items: center;
}

#ageGroupCategoryChart {
  width: 100%;
  height: 100%;
}

.sales-count {
  color: #67C23A;
  font-weight: bold;
}
</style>

5.5 路由配置

import { createRouter, createWebHistory } from 'vue-router';
import HomeView from '../views/HomeView.vue';
import CategorySalesDetailView from '../views/CategorySalesDetailView.vue';
import CategoryTotalSalesView from '../views/CategoryTotalSalesView.vue';
import PaymentMethodTotalSalesView from '../views/PaymentMethodTotalSalesView.vue';
import GenderSalesDetailView from '../views/GenderSalesDetailView.vue';
import AgeGroupCategorySalesView from '../views/AgeGroupCategorySalesView.vue';
import AgeGroupCategoryTotalSalesView from '../views/AgeGroupCategoryTotalSalesView.vue';
import GenderCategorySalesCountView from '../views/GenderCategorySalesCountView.vue';
import GenderCategoryTotalSalesView from '../views/GenderCategoryTotalSalesView.vue';

const routes = [
  {
    path: '/',
    name: 'home',
    component: HomeView
  },
  {
    path: '/category-sales-detail',
    name: 'category-sales-detail',
    component: CategorySalesDetailView
  },
  {
    path: '/category-total-sales',
    name: 'category-total-sales',
    component: CategoryTotalSalesView
  },
  {
    path: '/payment-method-total-sales',
    name: 'payment-method-total-sales',
    component: PaymentMethodTotalSalesView
  },
  {
    path: '/gender-sales-detail',
    name: 'gender-sales-detail',
    component: GenderSalesDetailView
  },
  {
    path: '/age-group-category-sales',
    name: 'age-group-category-sales',
    component: AgeGroupCategorySalesView
  },
  {
    path: '/age-group-category-total-sales',
    name: 'age-group-category-total-sales',
    component: AgeGroupCategoryTotalSalesView
  },
  {
    path: '/gender-category-sales-count',
    name: 'gender-category-sales-count',
    component: GenderCategorySalesCountView
  },
  {
    path: '/gender-category-total-sales',
    name: 'gender-category-total-sales',
    component: GenderCategoryTotalSalesView
  }
];

const router = createRouter({
  history: createWebHistory(),
  routes
});

export default router;

六、项目部署与运行

6.1 环境要求

• JDK 1.8+
• Maven 3.6+
• Node.js 20.19.0+
• Hadoop环境（用于MapReduce作业）

6.2 MapReduce模块运行

cd customerShopping-mapreduce
mvn clean package
hadoop jar target/customerShopping-mapreduce-1.0-SNAPSHOT.jar org.customerShopping.driver.ShoppingAnalysisDriver

6.3 API服务模块运行

cd customerShopping-api
mvn clean package
java -jar target/customerShopping-api-1.0-SNAPSHOT.jar

6.4 前端仪表板运行

cd customerShopping-dashboard
npm install
npm run dev

6.5 项目目录结构

customerShoppingAnalysis/
├── customerShopping-api/           # API服务模块
│   ├── pom.xml
│   └── src/
├── customerShopping-dashboard/     # 前端仪表板模块
│   ├── package.json
│   └── src/
├── customerShopping-mapreduce/     # MapReduce数据处理模块
│   ├── pom.xml
│   └── src/
├── data/                           # 数据目录
│   └── customer_shopping_data.csv
├── docs/                           # 文档目录
├── output/                         # MapReduce输出目录
│   ├── category_detail_sales/
│   ├── category_sales/
│   ├── payment_method_sales/
│   ├── gender_detail_sales/
│   ├── age_group_category_sales/
│   ├── age_group_category_total_sales/
│   ├── gender_category_sales_count/
│   └── gender_category_total_sales/
└── pom.xml

七、数据分析功能展示

7.1 商品类别分析

• 商品类别总销售额：统计每个商品类别的总销售额
• 商品类别销售详情：统计每个商品类别的总销售额、平均价格和总销售数量

业务价值：帮助企业了解哪些商品类别销售最好，哪些商品类别的平均价格最高

7.2 支付方式分析

• 支付方式总销售额：统计每种支付方式的总销售额

业务价值：了解客户偏好的支付方式，为支付方式优化提供依据

7.3 性别分析

• 性别销售详情：统计不同性别的总销售额、平均价格和总销售数量

业务价值：了解不同性别客户的消费行为差异

7.4 年龄组分析

• 年龄组类别销售数量：统计不同年龄段在各商品类别的销售数量
• 年龄组类别总销售额：统计不同年龄段在各商品类别的消费总额

业务价值：了解不同年龄段客户对各类别商品的偏好和消费能力

7.5 性别与商品类别交叉分析

• 性别商品类别销售数量：统计不同性别在各商品类别的销售数量
• 性别商品类别总销售额：统计不同性别在各商品类别的消费总额

业务价值：了解不同性别客户对各类别商品的偏好和消费能力

八、技术亮点

8.1 MapReduce多维度分析

系统实现了8种不同的MapReduce分析任务，每种任务对应不同的Mapper和Reducer实现，实现了数据的并行处理和高效计算。

8.2 自定义Hadoop数据类型

实现了自定义的Writable类型ShoppingSalesWritable，用于存储销售详情（总销售额、平均价格、总数量），支持数据的序列化和反序列化。

8.3 RESTful API设计

基于Spring Boot实现了RESTful API接口，提供标准化的数据查询服务，支持跨域访问。

8.4 数据可视化

使用ECharts实现了多种图表类型，包括柱状图、饼图、热力图等，直观展示数据分析结果。

8.5 响应式设计

前端采用Vue 3 Composition API，实现了响应式数据绑定和组件化开发。

九、项目总结与展望

9.1 项目总结

本项目实现了一个完整的电商购物数据分析平台，通过MapReduce处理海量购物数据，提供RESTful API查询接口，并通过Web界面可视化展示分析结果。系统支持多维度的数据分析，包括商品类别、支付方式、性别、年龄组等维度，帮助企业更好地了解客户行为和销售趋势。

9.2 未来展望

1. 数据实时分析：引入实时流处理技术，如Apache Flink，实现销售数据的实时分析
2. 机器学习集成：结合机器学习算法，实现销售预测、客户分群等高级分析功能
3. 用户权限管理：增加用户认证和授权功能，实现数据分析结果的权限控制
4. 移动端支持：开发移动端应用，方便用户随时随地查看分析结果
5. 更多分析维度：增加时间维度、地区维度等更多分析维度，提供更全面的数据分析功能

十、参考资料

• Hadoop官方文档
• Spring Boot官方文档
• Vue.js官方文档
• ECharts官方文档
• Element Plus官方文档

---

项目源码地址
作者：大数据基础
发布时间：2026年